Oct 30 12:56:26.683928 ip-10-0-130-180 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 12:56:27.067122 ip-10-0-130-180 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:56:27.067122 ip-10-0-130-180 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 12:56:27.067122 ip-10-0-130-180 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:56:27.067122 ip-10-0-130-180 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:56:27.072504 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072421    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:27.072504 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072503    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:27.072504 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072507    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072510    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072513    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072516    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072518    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072521    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072524    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072527    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072529    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072532    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072535    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072538    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072540    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072543    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072545    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072548    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072550    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072553    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072555    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072558    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:27.072597 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072560    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072563    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072566    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072568    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072572    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072574    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072577    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072580    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072583    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072586    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072588    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072591    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072593    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072596    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072599    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072602    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072604    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072607    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072610    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:27.073093 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072612    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072616    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072618    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072627    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072630    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072633    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072649    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072652    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072655    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072658    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072660    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072663    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072665    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072668    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072670    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072673    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072675    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072678    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072681    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072683    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:27.073547 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072686    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072689    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072692    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072694    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072697    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072699    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072704    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072709    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072712    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072715    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072718    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072721    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072723    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072726    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072728    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072731    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072733    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072737    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072739    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:27.074036 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072742    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072745    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072747    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072750    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072752    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072755    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072757    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072760    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072764    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072768    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072771    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072773    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072775    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072778    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072781    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072784    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072787    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072789    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072792    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:27.074491 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072795    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072798    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072800    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072803    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072806    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072808    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.072812    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073583    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073589    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073593    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073596    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073601    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073613    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073617    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073621    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073624    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073627    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073630    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073632    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:27.074957 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073635    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073651    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073654    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073656    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073659    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073662    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073665    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073668    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073670    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073673    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073676    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073679    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073681    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073684    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073687    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073689    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073692    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073694    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073697    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:27.075407 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073700    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073702    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073704    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073707    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073709    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073712    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073714    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073723    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073726    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073728    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073731    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073733    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073736    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073746    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073750    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073752    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073755    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073758    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073760    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073764    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:27.075863 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073768    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073770    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073773    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073775    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073778    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073780    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073783    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073786    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073789    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073791    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073794    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073797    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073799    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073802    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073804    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073808    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073812    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073814    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073817    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:27.076344 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073820    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073829    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073832    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073834    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073837    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073839    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073842    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073845    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073847    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073849    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073852    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073854    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073857    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073860    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073863    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073865    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073868    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073871    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073873    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073876    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:27.076798 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073879    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073881    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073884    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073886    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073889    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073892    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073894    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073896    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073899    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073902    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073905    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073908    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073910    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073912    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073915    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.073918    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074386    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074390    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074393    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074396    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:27.077271 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074399    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074403    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074407    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074409    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074412    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074415    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074418    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074421    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074423    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074426    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074429    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074434    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074437    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074440    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074442    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074445    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074448    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074451    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074453    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:27.077762 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074456    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074459    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074462    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074466    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074470    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074473    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074476    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074479    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074481    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074484    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074486    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074489    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074491    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074494    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074497    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074500    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074502    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074505    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074508    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074511    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:27.078234 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074513    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074516    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074518    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074521    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074524    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074527    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074529    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074532    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074534    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074537    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074539    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074543    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074545    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074548    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074550    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074553    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074555    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074558    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074560    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074563    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:27.078713 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074566    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074568    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074571    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074574    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074577    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074579    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074582    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074584    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074587    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074590    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074592    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074595    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074598    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074601    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074604    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074606    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074609    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074612    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074614    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:27.079892 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074617    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074620    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074622    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074625    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074629    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074632    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074634    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074649    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074652    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074655    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074658    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074660    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074663    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074665    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074668    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074671    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074674    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074676    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074679    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:27.080361 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074681    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074684    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074687    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074690    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.074692    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075827    2583 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075838    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075844    2583 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075849    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075853    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075857    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075861    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075866    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075869    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075873    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075876    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075880    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075884    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075887    2583 flags.go:64] FLAG: --cgroup-root=""
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075890    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075893    2583 flags.go:64] FLAG: --client-ca-file=""
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075895    2583 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075898    2583 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 12:56:27.080846 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075903    2583 flags.go:64] FLAG: --cluster-domain=""
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075906    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075909    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075912    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075916    2583 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075920    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075924    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075927    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075930    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075933    2583 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075936    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075939    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075942    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075945    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075949    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075952    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075956    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075959    2583 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075962    2583 flags.go:64] FLAG: --enable-server="true"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075965    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075969    2583 flags.go:64] FLAG: --event-burst="100"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075972    2583 flags.go:64] FLAG: --event-qps="50"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075975    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075978    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 12:56:27.081382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075981    2583 flags.go:64] FLAG: --eviction-hard=""
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075986    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075989    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075992    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075995    2583 flags.go:64] FLAG: --eviction-soft=""
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.075998    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076001    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076004    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076007    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076010    2583 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076013    2583 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076016    2583 flags.go:64] FLAG: --feature-gates=""
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076020    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076023    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076027    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076030    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076033    2583 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076036    2583 flags.go:64] FLAG: --help="false"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076039    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076042    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076045    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076047    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076051    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 12:56:27.082012 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076054    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076057    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076060    2583 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076065    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076068    2583 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076071    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076074    2583 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076077    2583 flags.go:64] FLAG: --kube-reserved=""
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076080    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076083    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076086    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076089    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076092    2583 flags.go:64] FLAG: --lock-file=""
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076095    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076098    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076101    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076106    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076109    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076112    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076115    2583 flags.go:64] FLAG: --logging-format="text"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076118    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076121    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076124    2583 flags.go:64] FLAG: --manifest-url=""
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076127    2583 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076132    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 12:56:27.082571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076135    2583 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076139    2583 flags.go:64] FLAG: --max-pods="110"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076142    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076145    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076148    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076151    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076154    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076157    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076159    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076166    2583 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076171    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076174    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076177    2583 flags.go:64] FLAG: --pod-cidr=""
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076180    2583 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076183    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076186    2583 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076189    2583 flags.go:64] FLAG: --port="10250"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076192    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076195    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04b9594dc22ebcc08"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076198    2583 flags.go:64] FLAG: --qos-reserved=""
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076201    2583 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076203    2583 flags.go:64] FLAG: --register-node="true"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076206    2583 flags.go:64] FLAG: --register-with-taints=""
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076210    2583 flags.go:64] FLAG: --registry-burst="10"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076213    2583 flags.go:64] FLAG: --registry-qps="5"
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076216    2583 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 12:56:27.083286 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076218    2583 flags.go:64] FLAG: --reserved-memory=""
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076222    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076225    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076228    2583 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076231    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076234    2583 flags.go:64] FLAG: --runonce="false"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076237    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076240    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076243    2583 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076246    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076249    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076252    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076255    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076259    2583 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076261    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076264    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076269    2583 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076273    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076276    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076279    2583 flags.go:64] FLAG: --system-cgroups=""
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076282    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076288    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076291    2583 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076294    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076298    2583 flags.go:64] FLAG: --tls-min-version=""
Oct 30 12:56:27.083922 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076301    2583 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076304    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076307    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076310    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076313    2583 flags.go:64] FLAG: --v="2"
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076317    2583 flags.go:64] FLAG: --version="false"
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076321    2583 flags.go:64] FLAG: --vmodule=""
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076326    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076329    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076473    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076477    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076480    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076482    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076485    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076488    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076491    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076494    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076496    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076499    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076502    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076504    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:27.084510 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076507    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076510    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076514    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076517    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076520    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076523    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076526    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076529    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076532    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076534    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076537    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076539    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076542    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076545    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076547    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076551    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076555    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076558    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076561    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:27.085083 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076564    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076567    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076569    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076572    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076574    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076577    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076579    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076582    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076585    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076588    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076590    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076593    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076595    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076598    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076601    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076604    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076608    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076611    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076613    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076616    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:27.085560 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076619    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076621    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076626    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076629    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076633    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076649    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076652    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076655    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076658    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076660    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076663    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076666    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076668    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076671    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076673    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076676    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076679    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076681    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076684    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:27.086048 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076687    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076690    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076693    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076696    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076698    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076701    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076704    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076706    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076710    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076714    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076716    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076719    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076722    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076725    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076727    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076730    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076733    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076735    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076738    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076740    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:27.086502 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076743    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076746    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076748    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076751    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076753    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076756    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076758    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076761    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076763    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076766    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076768    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076771    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076773    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076776    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076779    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076781    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:27.086990 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.076808    2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076845    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076849    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076852    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076856    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076859    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076863    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076866    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076869    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076871    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076874    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076877    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076879    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076882    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076884    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076887    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076890    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076892    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076895    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:27.087363 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076897    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076901    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076904    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076908    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076911    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076914    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076916    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076919    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076921    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076924    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076926    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076929    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076932    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076935    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076937    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076940    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076943    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076947    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076950    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076954    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:27.087821 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076957    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076960    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076962    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076964    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076967    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076970    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076972    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076977    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076980    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076984    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076987    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076989    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076992    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076995    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.076997    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077000    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077002    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077005    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077007    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:27.088317 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077010    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077013    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077015    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077017    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077020    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077023    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077026    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077028    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077031    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077033    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077037    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077040    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077044    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077047    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077049    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077052    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077054    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077057    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077059    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:27.088817 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077062    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077065    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077067    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077070    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077072    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077075    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077077    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077081    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077083    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077086    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077088    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077091    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077093    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077096    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077099    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077101    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077104    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077106    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077109    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077111    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:27.089275 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077114    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077117    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077119    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077124    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077126    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077130    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077133    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077136    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077138    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.077141    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.084156    2583 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.084171    2583 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.084187    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.084192    2583 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084249    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084254    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084258    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084261    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084264    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:27.089764 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084266    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084269    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084272    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084275    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084277    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084280    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084283    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084286    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084288    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084291    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084294    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084296    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084299    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084302    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084305    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084308    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084310    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084313    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084315    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:27.090218 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084318    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084322    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084326    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084329    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084332    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084335    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084338    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084341    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084343    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084346    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084350    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084354    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084358    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084360    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084363    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084366    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084369    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084371    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084374    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:27.090710 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084377    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084379    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084381    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084384    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084387    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084389    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084391    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084394    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084397    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084400    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084402    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084405    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084407    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084410    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084412    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084415    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084417    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084420    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084423    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084426    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:27.091178 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084428    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084431    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084433    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084435    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084438    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084441    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084443    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084446    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084448    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084451    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084453    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084456    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084458    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084461    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084463    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084466    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084468    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084470    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084473    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:27.091650 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084475    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084478    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084481    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084484    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084486    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084489    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084491    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084494    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084496    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084499    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084501    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084504    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084507    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084510    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084512    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084515    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084518    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084521    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084524    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084526    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:27.092094 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084529    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084531    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084534    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084536    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.084564    2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084716    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084721    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084724    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084727    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084731    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084733    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084736    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084738    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084741    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084744    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:27.092550 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084747    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084749    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084752    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084754    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084757    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084760    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084762    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084765    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084768    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084770    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084773    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084776    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084778    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084781    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084783    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084786    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084788    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084791    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084794    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084796    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:27.092975 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084798    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084801    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084803    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084806    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084808    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084811    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084813    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084816    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084818    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084821    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084823    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084825    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084828    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084831    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084833    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084836    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084838    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084840    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084843    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:27.093441 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084845    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084848    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084851    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084853    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084857    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084861    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084864    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084867    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084869    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084873    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084876    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084879    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084882    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084884    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084887    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084890    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084892    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084894    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084897    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:27.093953 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084900    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084902    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084905    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084907    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084909    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084912    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084915    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084917    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084920    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084923    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084925    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084927    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084930    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084932    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084935    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084938    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084940    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084943    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084945    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084948    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:27.095254 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084950    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084953    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084955    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084958    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084960    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084962    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084965    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084968    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084970    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084973    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084975    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084978    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084980    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084983    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084985    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084988    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084990    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:27.096095 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.084993    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.085016    2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085048    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085051    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085058    2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085061    2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085064    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085066    2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085068    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085071    2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085074    2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085076    2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085079    2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085081    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085084    2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085087    2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:56:27.096551 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085090    2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085092    2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085095    2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085097    2583 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085100    2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085102    2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085105    2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085108    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085111    2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085113    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085116    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085118    2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085121    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085124    2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085126    2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085129    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085131    2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085134    2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085136    2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:56:27.097011 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085139    2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085141    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085144    2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085147    2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085149    2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085152    2583 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085155    2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085158    2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085160    2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085162    2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085165    2583 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085167    2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085171    2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085173    2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085176    2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085178    2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085180    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085183    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085186    2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085188    2583 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:56:27.097487 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085191    2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085193    2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085196    2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085198    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085200    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085203    2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085205    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085208    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085210    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085213    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085215    2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085217    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085220    2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085222    2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085225    2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085228    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085230    2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085232    2583 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085235    2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:56:27.097990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085238    2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085240    2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085243    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085245    2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085249    2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085253    2583 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085257    2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085260    2583 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085262    2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085266    2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085270    2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085273    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085276    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085278    2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085281    2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085283    2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085286    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085288    2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085290    2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:56:27.098426 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085293    2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085296    2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085298    2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085300    2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085303    2583 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085305    2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085308    2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085311    2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085313    2583 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085315    2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085318    2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085321    2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085324    2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085326    2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:27.085329    2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.085378    2583 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.087842    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.088907    2583 server.go:1014] "Starting client certificate rotation"
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.089006    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 12:56:27.098976 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.089039    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 12:56:27.099421 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.091089    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:56:27.099421 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.096793    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:56:27.108377 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.108361    2583 log.go:25] "Validated CRI v1 runtime API"
Oct 30 12:56:27.112373 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.112356    2583 log.go:25] "Validated CRI v1 image API"
Oct 30 12:56:27.113204 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.113178    2583 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 12:56:27.116736 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.116711    2583 fs.go:135] Filesystem UUIDs: map[73a38cac-c335-411e-808a-b3bbe02cef4f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9c053e75-2ec8-4c07-b296-7bbbd1126cda:/dev/nvme0n1p4]
Oct 30 12:56:27.116798 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.116735    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 12:56:27.122876 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.122756    2583 manager.go:217] Machine: {Timestamp:2026-10-30 12:56:27.121302058 +0000 UTC m=+0.351026922 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3130193 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20920f0b2a032fd856ae91ba40243c SystemUUID:ec20920f-0b2a-032f-d856-ae91ba40243c BootID:c055694b-75f1-475b-a05a-5599528bee6b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c9:bd:90:57 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c9:bd:90:57 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:7b:33:07:c3:0c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 12:56:27.122876 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.122871    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 12:56:27.123029 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.122947    2583 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 12:56:27.124203 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.124180    2583 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 12:56:27.124373 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.124204    2583 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-180.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 12:56:27.124458 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.124378    2583 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 12:56:27.124458 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.124391    2583 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 12:56:27.124458 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.124438    2583 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:56:27.124566 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.124464    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:56:27.125813 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.125799    2583 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 12:56:27.130617 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.130600    2583 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 12:56:27.130795 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.130784    2583 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 12:56:27.131652 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.131630    2583 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 12:56:27.131711 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.131657    2583 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 12:56:27.131711 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.131669    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 12:56:27.131711 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.131678    2583 kubelet.go:405] "Adding apiserver pod source"
Oct 30 12:56:27.131711 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.131688    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 12:56:27.132000 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.131987    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:56:27.134662 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.134622    2583 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 12:56:27.141339 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.141327    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:56:27.142200 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.142188    2583 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 12:56:27.142243 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.142218    2583 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 12:56:27.142373 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.142362    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 12:56:27.142404 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.142386    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 12:56:27.142404 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.142392    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 12:56:27.142404 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.142399    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 12:56:27.142404 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.142405    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 12:56:27.142510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.142412    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 12:56:27.143099 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.143090    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 12:56:27.143133 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.143100    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 12:56:27.143133 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.143107    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 12:56:27.143133 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.143115    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 12:56:27.143133 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.143123    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 12:56:27.143133 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.143132    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 12:56:27.144396 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.144387    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 12:56:27.144429 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.144400    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 12:56:27.148215 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.148202    2583 server.go:1263] "Started kubelet"
Oct 30 12:56:27.148294 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.148271    2583 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 12:56:27.148345 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.148283    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 12:56:27.148377 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.148365    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 12:56:27.152749 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.152729    2583 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 12:56:27.152873 ip-10-0-130-180 systemd[1]: Started Kubernetes Kubelet.
Oct 30 12:56:27.156295 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.156275    2583 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 12:56:27.159606 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.159585    2583 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 12:56:27.163409 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.163390    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 12:56:27.164820 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.164807    2583 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 12:56:27.164960 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.164937    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:27.165589 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165570    2583 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 12:56:27.165589 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165569    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 12:56:27.165589 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165591    2583 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 12:56:27.165806 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165621    2583 factory.go:55] Registering systemd factory
Oct 30 12:56:27.165806 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165653    2583 factory.go:223] Registration of the systemd container factory successfully
Oct 30 12:56:27.165806 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165747    2583 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 12:56:27.165806 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165755    2583 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 12:56:27.165924 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165885    2583 factory.go:153] Registering CRI-O factory
Oct 30 12:56:27.165924 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165900    2583 factory.go:223] Registration of the crio container factory successfully
Oct 30 12:56:27.165982 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165941    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 12:56:27.165982 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165959    2583 factory.go:103] Registering Raw factory
Oct 30 12:56:27.165982 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.165974    2583 manager.go:1199] Started watching for new ooms in manager
Oct 30 12:56:27.167071 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.167057    2583 manager.go:319] Starting recovery of all containers
Oct 30 12:56:27.176345 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.176327    2583 manager.go:324] Recovery completed
Oct 30 12:56:27.181121 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.181109    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:27.182754 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.182732    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:27.182826 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.182768    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:27.183045 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.183028    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:27.183762 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.183745    2583 cpu_manager.go:226] "Starting" policy="none"
Oct 30 12:56:27.183762 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.183761    2583 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 12:56:27.183881 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.183781    2583 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 12:56:27.183881 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.183798    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:27.184754 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.184736    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:27.184824 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.184766    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:27.184824 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.184777    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:27.184915 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.184843    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-180.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 12:56:27.184959 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.184937    2583 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 12:56:27.186356 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.184833    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-180.ec2.internal.18e34ff4439708a0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-180.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-180.ec2.internal,},FirstTimestamp:2026-10-30 12:56:27.14817552 +0000 UTC m=+0.377900382,LastTimestamp:2026-10-30 12:56:27.14817552 +0000 UTC m=+0.377900382,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-180.ec2.internal,}"
Oct 30 12:56:27.187898 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.187879    2583 policy_none.go:50] "Start"
Oct 30 12:56:27.187898 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.187899    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 12:56:27.188036 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.187910    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 12:56:27.189948 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.189937    2583 policy_none.go:44] "Start"
Oct 30 12:56:27.190845 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.190830    2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 12:56:27.190888 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.190865    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7w5rm"
Oct 30 12:56:27.200579 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.200564    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7w5rm"
Oct 30 12:56:27.210377 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.210338    2583 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-180.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 12:56:27.210377 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.210357    2583 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 12:56:27.210514 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.210402    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-180.ec2.internal.18e34ff445a6a02f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-180.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-180.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-180.ec2.internal,},FirstTimestamp:2026-10-30 12:56:27.182751791 +0000 UTC m=+0.412476660,LastTimestamp:2026-10-30 12:56:27.182751791 +0000 UTC m=+0.412476660,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-180.ec2.internal,}"
Oct 30 12:56:27.210514 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.210471    2583 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 12:56:27.216273 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.216243    2583 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 12:56:27.219676 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.219632    2583 manager.go:341] "Starting Device Plugin manager"
Oct 30 12:56:27.219758 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.219677    2583 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 12:56:27.219758 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.219688    2583 server.go:87] "Starting device plugin registration server"
Oct 30 12:56:27.219957 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.219944    2583 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 12:56:27.220017 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.219959    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 12:56:27.220081 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.220065    2583 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 12:56:27.220191 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.220179    2583 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 12:56:27.220191 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.220189    2583 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 12:56:27.220715 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.220695    2583 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 12:56:27.220810 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.220751    2583 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:27.302673 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.302624    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 12:56:27.304799 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.303783    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 12:56:27.304799 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.303804    2583 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 12:56:27.304799 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.303831    2583 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 12:56:27.304799 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.303869    2583 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 12:56:27.318767 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.318680    2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:27.320754 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.320736    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:27.321582 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.321565    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:27.321718 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.321596    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:27.321718 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.321610    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:27.321718 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.321663    2583 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.340750 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.340733    2583 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.340870 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.340758    2583 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-180.ec2.internal\": node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:27.393065 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.393037    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:27.404512 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.404482    2583 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"]
Oct 30 12:56:27.404600 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.404546    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:27.405978 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.405963    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:27.406058 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.405989    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:27.406058 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.406001    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:27.408369 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.408356    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:27.408529 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.408515    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.408576 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.408545    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:27.409841 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.409821    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:27.409924 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.409848    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:27.409924 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.409860    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:27.409924 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.409825    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:27.409924 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.409893    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:27.409924 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.409911    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:27.412077 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.412062    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.412148 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.412087    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:27.412819 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.412804    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:27.412915 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.412826    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:27.412915 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.412835    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:27.431383 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.431361    2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-180.ec2.internal\" not found" node="ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.434131 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.434116    2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-180.ec2.internal\" not found" node="ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.467475 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.467453    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.467555 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.467479    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.467555 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.467496    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbd004dde5b9ed65a173b78115c079e0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-180.ec2.internal\" (UID: \"dbd004dde5b9ed65a173b78115c079e0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.493396 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.493375    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:27.568120 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.568093    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.568234 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.568127    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.568234 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.568144    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbd004dde5b9ed65a173b78115c079e0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-180.ec2.internal\" (UID: \"dbd004dde5b9ed65a173b78115c079e0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.568234 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.568203    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.568234 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.568204    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.568350 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.568228    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbd004dde5b9ed65a173b78115c079e0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-180.ec2.internal\" (UID: \"dbd004dde5b9ed65a173b78115c079e0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.593857 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.593813    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:27.694122 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.694096    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:27.734229 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.734208    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.737358 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:27.737336    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:27.795121 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.794962    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:27.895544 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.895472    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:27.995973 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:27.995946    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:28.088585 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:28.088555    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 12:56:28.089033 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:28.088699    2583 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 12:56:28.096839 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:28.096815    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:29.955871 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:29.955841    2583 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:29.978538 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:29.978512    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:29.987681 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:29.987651    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 12:51:28 +0000 UTC" deadline="2028-07-03 18:41:57.16778104 +0000 UTC"
Oct 30 12:56:29.987681 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:29.987679    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14693h45m27.180105477s"
Oct 30 12:56:30.002655 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.002629    2583 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:30.010625 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.010610    2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:56:30.014113 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.014093    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:56:30.014196 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.014125    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:56:30.014196 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.014139    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:56:30.065399 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:30.065374    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd004dde5b9ed65a173b78115c079e0.slice/crio-914daae99f98a21338ff4a5f32aba5ef1c8376b8411f53722835adbe3af5a601 WatchSource:0}: Error finding container 914daae99f98a21338ff4a5f32aba5ef1c8376b8411f53722835adbe3af5a601: Status 404 returned error can't find the container with id 914daae99f98a21338ff4a5f32aba5ef1c8376b8411f53722835adbe3af5a601
Oct 30 12:56:30.066375 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:30.066359    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b3f18c352b4e52d3d8a287983d93c2b.slice/crio-a6d797fb42450ffe205b46017cda83bb21b1a4e8e3ea7d054661bcca72bb445b WatchSource:0}: Error finding container a6d797fb42450ffe205b46017cda83bb21b1a4e8e3ea7d054661bcca72bb445b: Status 404 returned error can't find the container with id a6d797fb42450ffe205b46017cda83bb21b1a4e8e3ea7d054661bcca72bb445b
Oct 30 12:56:30.070488 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.070465    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:56:30.078648 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:30.078630    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:30.087587 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.087546    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal" event={"ID":"dbd004dde5b9ed65a173b78115c079e0","Type":"ContainerStarted","Data":"914daae99f98a21338ff4a5f32aba5ef1c8376b8411f53722835adbe3af5a601"}
Oct 30 12:56:30.088618 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.088600    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal" event={"ID":"9b3f18c352b4e52d3d8a287983d93c2b","Type":"ContainerStarted","Data":"a6d797fb42450ffe205b46017cda83bb21b1a4e8e3ea7d054661bcca72bb445b"}
Oct 30 12:56:30.178773 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:30.178685    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:30.279337 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:30.279300    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:30.379936 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:30.379901    2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:56:30.430001 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.429928    2583 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:30.446035 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.446003    2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:30.470197 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.470163    2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:56:30.470925 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.470902    2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:56:30.491744 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.491635    2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:56:30.914343 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.914269    2583 apiserver.go:52] "Watching apiserver"
Oct 30 12:56:30.944721 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.944691    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 12:56:30.958907 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.958881    2583 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 12:56:30.959238 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.959210    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-k6rf5","openshift-network-diagnostics/network-check-target-xc6hk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal","openshift-multus/multus-additional-cni-plugins-97p84","openshift-ovn-kubernetes/ovnkube-node-nzs4p","kube-system/global-pull-secret-syncer-gkrqb","openshift-cluster-node-tuning-operator/tuned-5gn6d","openshift-network-operator/iptables-alerter-sdmv9","kube-system/konnectivity-agent-c56xj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz","openshift-dns/node-resolver-rlghf","openshift-multus/network-metrics-daemon-xtz6h","kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal","openshift-image-registry/node-ca-t7zb4"]
Oct 30 12:56:30.961823 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.961799    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:30.964907 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.964885    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:30.967092 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.967070    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:30.970093 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.970073    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:30.971666 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.971645    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:30.973900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.973882    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:30.974003 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:30.973970    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:30.978280 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.976272    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:30.978881 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.978865    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:30.979049 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.979032    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-25fk6\""
Oct 30 12:56:30.979488 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.979472    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:56:30.979571 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.979516    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 12:56:30.980824 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.980801    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 12:56:30.980941 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.980922    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 12:56:30.981065 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.981050    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dz42j\""
Oct 30 12:56:30.981228 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.981204    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:30.981388 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:30.981366    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:30.983873 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.983855    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:30.986419 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.986396    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:30.987552 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.987535    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cxt4s\""
Oct 30 12:56:30.987999 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.987972    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-94phb\""
Oct 30 12:56:30.988370 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.988345    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:30.988455 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:30.988417    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:30.988906 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.988887    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xt92t\""
Oct 30 12:56:30.990623 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.990601    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 12:56:30.990718 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.990642    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 12:56:30.990718 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.990661    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 12:56:30.990828 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.990814    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 12:56:30.991716 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.991698    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 12:56:30.993622 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.992094    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 12:56:30.993622 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.992152    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 12:56:30.994632 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.994472    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 12:56:30.994632 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.994523    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 12:56:30.994632 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.994557    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 12:56:30.994632 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.994579    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 12:56:30.994632 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:30.994609    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 12:56:31.001972 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.001954    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 12:56:31.002063 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.002029    2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 12:56:31.002121 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.002063    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 12:56:31.002177 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.002158    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6lrfm"
Oct 30 12:56:31.008794 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.008777    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6lrfm"
Oct 30 12:56:31.048121 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.048100    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 12:56:31.059260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059231    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39289142-2079-4b45-ba09-8e06aed12276-agent-certs\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:31.059408 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059275    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.059408 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059325    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.059408 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059369    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v5gg\" (UniqueName: \"kubernetes.io/projected/853421d2-ef94-4cbd-b251-1f4d82bfb45e-kube-api-access-2v5gg\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.059566 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059425    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-netns\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.059566 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059475    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-system-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.059566 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059521    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-conf-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.059566 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059560    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kh5j\" (UniqueName: \"kubernetes.io/projected/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-kube-api-access-8kh5j\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.059757 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059609    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgt5q\" (UniqueName: \"kubernetes.io/projected/f4471adf-01e8-47ae-8056-c670541a735c-kube-api-access-bgt5q\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.059757 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059639    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.059757 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059664    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-etc-selinux\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.059757 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853421d2-ef94-4cbd-b251-1f4d82bfb45e-hosts-file\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.059948 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059741    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-env-overrides\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.059948 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059802    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-bin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.059948 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059827    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-multus-daemon-config\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.059948 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059851    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.059948 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-multus\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.059948 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059937    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-slash\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059962    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-node-log\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.059987    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-k8s-cni-cncf-io\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060011    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-etc-kubernetes\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060035    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysconfig\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060061    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-system-cni-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060086    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2926b50e-e07d-445d-bbec-eab0005a59bf-serviceca\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060111    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-var-lib-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060134    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-systemd\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060168    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060194    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.060260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060218    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-multus-certs\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060327    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-lib-modules\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060367    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060436    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-sys-fs\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060469    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4c5v\" (UniqueName: \"kubernetes.io/projected/1062c6b4-6828-4e92-92a5-c78eccb1fe26-kube-api-access-c4c5v\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060500    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-kubelet\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060525    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7nwx\" (UniqueName: \"kubernetes.io/projected/0adce241-e2df-40ec-8976-341e00ebc074-kube-api-access-v7nwx\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060556    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-kubernetes\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-var-lib-kubelet\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060602    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-log-socket\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060630    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-dbus\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060654    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-iptables-alerter-script\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060678    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-kubelet\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060703    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-etc-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060727    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqs7p\" (UniqueName: \"kubernetes.io/projected/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kube-api-access-jqs7p\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.060859 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-sys\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060779    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-binary-copy\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39289142-2079-4b45-ba09-8e06aed12276-konnectivity-ca\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060828    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-bin\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060855    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-kubelet-config\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060880    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-socket-dir-parent\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.060904    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-modprobe-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061032    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-conf\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061063    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-ovn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061091    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061117    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-tmp\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061149    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-device-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061175    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853421d2-ef94-4cbd-b251-1f4d82bfb45e-tmp-dir\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-host-slash\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061225    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-cnibin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061249    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-cni-binary-copy\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.061616 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061273    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-netns\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061315    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-registration-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061341    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lh9t\" (UniqueName: \"kubernetes.io/projected/2926b50e-e07d-445d-bbec-eab0005a59bf-kube-api-access-8lh9t\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061364    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-config\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061388    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/162db930-0919-4366-ac55-b8e647cb714b-ovn-node-metrics-cert\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061415    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqzpn\" (UniqueName: \"kubernetes.io/projected/162db930-0919-4366-ac55-b8e647cb714b-kube-api-access-cqzpn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061440    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2926b50e-e07d-445d-bbec-eab0005a59bf-host\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061498    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-systemd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061526    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-run\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061552    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061577    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-netd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061605    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-script-lib\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l97l\" (UniqueName: \"kubernetes.io/projected/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-kube-api-access-5l97l\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-hostroot\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-tuned\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.062349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061729    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:31.063021 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061759    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-socket-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.063021 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061784    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-systemd-units\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.063021 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061809    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-host\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.063021 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.061836    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-cnibin\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.079944 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.079915    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 12:56:31.079944 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.079927    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 12:56:31.079944 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.079918    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:56:31.080252 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080078    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tzbll\""
Oct 30 12:56:31.080252 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080084    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-sp79l\""
Oct 30 12:56:31.080252 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080124    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 12:56:31.080252 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080135    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 12:56:31.080252 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080141    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 12:56:31.080252 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080146    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-wsrzk\""
Oct 30 12:56:31.080252 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080166    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 12:56:31.080252 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080196    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7gdbz\""
Oct 30 12:56:31.080623 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080260    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 12:56:31.080623 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.080341    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 12:56:31.162158 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162128    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:31.162158 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162174    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-sys-fs\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.162418 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4c5v\" (UniqueName: \"kubernetes.io/projected/1062c6b4-6828-4e92-92a5-c78eccb1fe26-kube-api-access-c4c5v\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:31.162418 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162221    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-kubelet\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.162418 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162242    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7nwx\" (UniqueName: \"kubernetes.io/projected/0adce241-e2df-40ec-8976-341e00ebc074-kube-api-access-v7nwx\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.162418 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.162260    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:31.162418 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162327    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-kubernetes\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.162418 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162327    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-kubelet\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.162418 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-kubernetes\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.162418 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.162364    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs podName:1062c6b4-6828-4e92-92a5-c78eccb1fe26 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:31.662336074 +0000 UTC m=+3.111412684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs") pod "network-metrics-daemon-xtz6h" (UID: "1062c6b4-6828-4e92-92a5-c78eccb1fe26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:31.162784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162425    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-var-lib-kubelet\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.162784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162458    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-log-socket\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.162784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-dbus\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:31.162784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162519    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-var-lib-kubelet\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.162784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162521    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-iptables-alerter-script\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.162784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162562    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-kubelet\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.162784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162603    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-kubelet\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.162784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162603    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-log-socket\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.162784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162619    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-etc-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.163097 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162829    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-dbus\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:31.163097 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162841    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-sys-fs\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.163097 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.162948    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-etc-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.163097 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163074    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqs7p\" (UniqueName: \"kubernetes.io/projected/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kube-api-access-jqs7p\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.163097 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163090    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-iptables-alerter-script\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-sys\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163127    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-sys\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163135    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-binary-copy\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163153    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39289142-2079-4b45-ba09-8e06aed12276-konnectivity-ca\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163169    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-bin\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-kubelet-config\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-socket-dir-parent\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163228    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-bin\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-modprobe-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163258    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-socket-dir-parent\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163265    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-kubelet-config\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163278    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-conf\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.163312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163316    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-ovn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163338    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163364    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-modprobe-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163381    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163391    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-tmp\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163397    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-ovn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-device-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163447    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853421d2-ef94-4cbd-b251-1f4d82bfb45e-tmp-dir\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163463    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-device-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163474    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-host-slash\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163484    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-conf\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-cnibin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163521    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-cni-binary-copy\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163536    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-host-slash\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-netns\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163575    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-registration-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163580    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-netns\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163629    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-cnibin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.163900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163643    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lh9t\" (UniqueName: \"kubernetes.io/projected/2926b50e-e07d-445d-bbec-eab0005a59bf-kube-api-access-8lh9t\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-config\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163690    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/162db930-0919-4366-ac55-b8e647cb714b-ovn-node-metrics-cert\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163712    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqzpn\" (UniqueName: \"kubernetes.io/projected/162db930-0919-4366-ac55-b8e647cb714b-kube-api-access-cqzpn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163716    2583 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163735    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163746    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853421d2-ef94-4cbd-b251-1f4d82bfb45e-tmp-dir\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163756    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2926b50e-e07d-445d-bbec-eab0005a59bf-host\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-systemd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163799    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-run\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163823    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163844    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39289142-2079-4b45-ba09-8e06aed12276-konnectivity-ca\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163850    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-binary-copy\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163849    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-netd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163886    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-netd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163896    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-script-lib\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163916    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2926b50e-e07d-445d-bbec-eab0005a59bf-host\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163924    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l97l\" (UniqueName: \"kubernetes.io/projected/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-kube-api-access-5l97l\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.164782 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163930    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-systemd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163948    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-hostroot\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163971    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-tuned\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.163978    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-run\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164005    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.164012    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164030    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-socket-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164128    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-cni-binary-copy\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164139    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-socket-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.164076    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret podName:c69d9666-d791-48fe-aad3-7a91f2ab2d2c nodeName:}" failed. No retries permitted until 2026-10-30 12:56:31.664044066 +0000 UTC m=+3.113120661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret") pod "global-pull-secret-syncer-gkrqb" (UID: "c69d9666-d791-48fe-aad3-7a91f2ab2d2c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164586    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-systemd-units\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164614    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-host\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164638    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-cnibin\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39289142-2079-4b45-ba09-8e06aed12276-agent-certs\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164712    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v5gg\" (UniqueName: \"kubernetes.io/projected/853421d2-ef94-4cbd-b251-1f4d82bfb45e-kube-api-access-2v5gg\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.165667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164748    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-registration-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-netns\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164785    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-system-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164812    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-conf-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164823    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-host\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164852    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-hostroot\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164860    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kh5j\" (UniqueName: \"kubernetes.io/projected/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-kube-api-access-8kh5j\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164877    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-netns\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164888    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgt5q\" (UniqueName: \"kubernetes.io/projected/f4471adf-01e8-47ae-8056-c670541a735c-kube-api-access-bgt5q\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164926    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164949    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-etc-selinux\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853421d2-ef94-4cbd-b251-1f4d82bfb45e-hosts-file\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165020    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-env-overrides\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165049    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-bin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-multus-daemon-config\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165102    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-multus\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.166506 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-slash\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165132    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-node-log\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165147    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-k8s-cni-cncf-io\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-etc-kubernetes\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165178    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysconfig\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165193    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-system-cni-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165209    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2926b50e-e07d-445d-bbec-eab0005a59bf-serviceca\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165228    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-var-lib-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165244    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-systemd\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165260    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165275    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165314    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-multus-certs\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165338    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-lib-modules\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165364    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.165632    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166124    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-system-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.164791    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-cnibin\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.167303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166215    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-systemd-units\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166266    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-conf-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166425    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-etc-selinux\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166492    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853421d2-ef94-4cbd-b251-1f4d82bfb45e-hosts-file\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166575    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166580    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166609    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166633    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysconfig\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166637    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-multus\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166655    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-bin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166684    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-slash\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166727    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-node-log\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166769    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-k8s-cni-cncf-io\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166810    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-etc-kubernetes\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166863    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-systemd\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.168361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.166905    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-system-cni-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.169096 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.168692    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-var-lib-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.169096 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.168712    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.169096 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.168786    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.169096 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.168832    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-multus-certs\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.169096 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.168926    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-lib-modules\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.169096 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.169018    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-env-overrides\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.169394 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.169150    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-script-lib\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.169394 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.169238    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.169394 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.169320    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-multus-daemon-config\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.169544 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.169506    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-tuned\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.170049 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.170030    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2926b50e-e07d-445d-bbec-eab0005a59bf-serviceca\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.170049 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.170040    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-tmp\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.170312 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.170278    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/162db930-0919-4366-ac55-b8e647cb714b-ovn-node-metrics-cert\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.170482 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.170328    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-config\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.173734 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.173630    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.174061 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.174041    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39289142-2079-4b45-ba09-8e06aed12276-agent-certs\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:31.188596 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.188561    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7nwx\" (UniqueName: \"kubernetes.io/projected/0adce241-e2df-40ec-8976-341e00ebc074-kube-api-access-v7nwx\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.189301 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.189267    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:31.189423 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.189312    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:31.189423 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.189328    2583 projected.go:196] Error preparing data for projected volume kube-api-access-srms2 for pod openshift-network-diagnostics/network-check-target-xc6hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:31.189423 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.189351    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4c5v\" (UniqueName: \"kubernetes.io/projected/1062c6b4-6828-4e92-92a5-c78eccb1fe26-kube-api-access-c4c5v\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:31.189423 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.189409    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2 podName:4faa20e7-dd05-4416-9b9e-ed52b91d388d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:31.689388771 +0000 UTC m=+3.138465374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-srms2" (UniqueName: "kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2") pod "network-check-target-xc6hk" (UID: "4faa20e7-dd05-4416-9b9e-ed52b91d388d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:31.191493 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.191186    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqs7p\" (UniqueName: \"kubernetes.io/projected/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kube-api-access-jqs7p\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.193995 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.193970    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqzpn\" (UniqueName: \"kubernetes.io/projected/162db930-0919-4366-ac55-b8e647cb714b-kube-api-access-cqzpn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.194361 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.194338    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kh5j\" (UniqueName: \"kubernetes.io/projected/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-kube-api-access-8kh5j\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.199930 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.199894    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgt5q\" (UniqueName: \"kubernetes.io/projected/f4471adf-01e8-47ae-8056-c670541a735c-kube-api-access-bgt5q\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.199930 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.199917    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l97l\" (UniqueName: \"kubernetes.io/projected/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-kube-api-access-5l97l\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.200053 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.199990    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lh9t\" (UniqueName: \"kubernetes.io/projected/2926b50e-e07d-445d-bbec-eab0005a59bf-kube-api-access-8lh9t\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.201218 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.201198    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v5gg\" (UniqueName: \"kubernetes.io/projected/853421d2-ef94-4cbd-b251-1f4d82bfb45e-kube-api-access-2v5gg\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.275693 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.275652    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:56:31.285572 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.285381    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:56:31.287990 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:31.287960    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f1ca79_0b5a_4a41_a574_e2bdb4a9a94d.slice/crio-ea7df2b853a22f5e1cdcfdf57a42f434621e5343f5be7d7cb43ecc05971a6d79 WatchSource:0}: Error finding container ea7df2b853a22f5e1cdcfdf57a42f434621e5343f5be7d7cb43ecc05971a6d79: Status 404 returned error can't find the container with id ea7df2b853a22f5e1cdcfdf57a42f434621e5343f5be7d7cb43ecc05971a6d79
Oct 30 12:56:31.293302 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.293271    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:56:31.296100 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:31.296043    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853421d2_ef94_4cbd_b251_1f4d82bfb45e.slice/crio-7b63fe8b9bff669d6d61a0846f7fa93f8ac602b5b3f2f2b049b537fa5b2efbf2 WatchSource:0}: Error finding container 7b63fe8b9bff669d6d61a0846f7fa93f8ac602b5b3f2f2b049b537fa5b2efbf2: Status 404 returned error can't find the container with id 7b63fe8b9bff669d6d61a0846f7fa93f8ac602b5b3f2f2b049b537fa5b2efbf2
Oct 30 12:56:31.303442 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:31.303406    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2926b50e_e07d_445d_bbec_eab0005a59bf.slice/crio-ce1ac0a68e3b02b8a5bec0ac63626c10ae12ae696f33bb32fd09b114e0270fbd WatchSource:0}: Error finding container ce1ac0a68e3b02b8a5bec0ac63626c10ae12ae696f33bb32fd09b114e0270fbd: Status 404 returned error can't find the container with id ce1ac0a68e3b02b8a5bec0ac63626c10ae12ae696f33bb32fd09b114e0270fbd
Oct 30 12:56:31.321428 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.321386    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:56:31.323596 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.323562    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:31.334000 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.333965    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:56:31.334336 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:31.334305    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4471adf_01e8_47ae_8056_c670541a735c.slice/crio-0f5ddbb728fbacf85b52607447cfa8da1ebc445a470d9cb53306a0bf051f5009 WatchSource:0}: Error finding container 0f5ddbb728fbacf85b52607447cfa8da1ebc445a470d9cb53306a0bf051f5009: Status 404 returned error can't find the container with id 0f5ddbb728fbacf85b52607447cfa8da1ebc445a470d9cb53306a0bf051f5009
Oct 30 12:56:31.335486 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:31.335460    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39289142_2079_4b45_ba09_8e06aed12276.slice/crio-e2439f6a602acc4de55b7b91cc20a90f0b8793364c7875738b90a8335828cecd WatchSource:0}: Error finding container e2439f6a602acc4de55b7b91cc20a90f0b8793364c7875738b90a8335828cecd: Status 404 returned error can't find the container with id e2439f6a602acc4de55b7b91cc20a90f0b8793364c7875738b90a8335828cecd
Oct 30 12:56:31.339405 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.339384    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:31.345403 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.345383    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:56:31.347464 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.347442    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k6rf5"
Oct 30 12:56:31.352329 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:31.350813    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d363e1_d64d_4ab4_a134_08ef9b8dd9f9.slice/crio-9a33816ef2cd30b219a540c24f03ef7342a59fb9412e1901e2c6f606ab997294 WatchSource:0}: Error finding container 9a33816ef2cd30b219a540c24f03ef7342a59fb9412e1901e2c6f606ab997294: Status 404 returned error can't find the container with id 9a33816ef2cd30b219a540c24f03ef7342a59fb9412e1901e2c6f606ab997294
Oct 30 12:56:31.355437 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:31.355416    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod162db930_0919_4366_ac55_b8e647cb714b.slice/crio-a1ca5baef5c7169c7da2ba7c009c908f20eca9358755d105faf0641fcf7f5cfd WatchSource:0}: Error finding container a1ca5baef5c7169c7da2ba7c009c908f20eca9358755d105faf0641fcf7f5cfd: Status 404 returned error can't find the container with id a1ca5baef5c7169c7da2ba7c009c908f20eca9358755d105faf0641fcf7f5cfd
Oct 30 12:56:31.364653 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:31.364632    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7b27fb_3f62_473c_89d0_ea4ced2bbe56.slice/crio-1c4544afd27f657628273ca69886f70932c2fa5a4ba34123ba5e5ce43203e091 WatchSource:0}: Error finding container 1c4544afd27f657628273ca69886f70932c2fa5a4ba34123ba5e5ce43203e091: Status 404 returned error can't find the container with id 1c4544afd27f657628273ca69886f70932c2fa5a4ba34123ba5e5ce43203e091
Oct 30 12:56:31.365526 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:56:31.365503    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0adce241_e2df_40ec_8976_341e00ebc074.slice/crio-71446ba892b17259700ac5db04ae23a27f073e8569b9bada878a8427adf79bc9 WatchSource:0}: Error finding container 71446ba892b17259700ac5db04ae23a27f073e8569b9bada878a8427adf79bc9: Status 404 returned error can't find the container with id 71446ba892b17259700ac5db04ae23a27f073e8569b9bada878a8427adf79bc9
Oct 30 12:56:31.448576 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.448549    2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:56:31.669165 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.668599    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:31.669165 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.668689    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:31.669165 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.668764    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:31.669165 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.668797    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:31.669165 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.668834    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret podName:c69d9666-d791-48fe-aad3-7a91f2ab2d2c nodeName:}" failed. No retries permitted until 2026-10-30 12:56:32.668817872 +0000 UTC m=+4.117894465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret") pod "global-pull-secret-syncer-gkrqb" (UID: "c69d9666-d791-48fe-aad3-7a91f2ab2d2c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:31.669165 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.668851    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs podName:1062c6b4-6828-4e92-92a5-c78eccb1fe26 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:32.668843675 +0000 UTC m=+4.117920263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs") pod "network-metrics-daemon-xtz6h" (UID: "1062c6b4-6828-4e92-92a5-c78eccb1fe26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:31.769484 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:31.769248    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:31.769631 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.769415    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:31.769631 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.769541    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:31.769631 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.769556    2583 projected.go:196] Error preparing data for projected volume kube-api-access-srms2 for pod openshift-network-diagnostics/network-check-target-xc6hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:31.769631 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:31.769607    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2 podName:4faa20e7-dd05-4416-9b9e-ed52b91d388d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:32.769592373 +0000 UTC m=+4.218668979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-srms2" (UniqueName: "kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2") pod "network-check-target-xc6hk" (UID: "4faa20e7-dd05-4416-9b9e-ed52b91d388d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:32.012317 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.012160    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:51:30 +0000 UTC" deadline="2028-05-11 12:58:56.326320093 +0000 UTC"
Oct 30 12:56:32.012317 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.012215    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13416h2m24.314109009s"
Oct 30 12:56:32.099871 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.099836    2583 generic.go:356] "Generic (PLEG): container finished" podID="9b3f18c352b4e52d3d8a287983d93c2b" containerID="543935082c87ae7e02833bd8ed70e93deac60a14b5c66d31c6eac65962bf54ea" exitCode=0
Oct 30 12:56:32.100037 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.099919    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal" event={"ID":"9b3f18c352b4e52d3d8a287983d93c2b","Type":"ContainerDied","Data":"543935082c87ae7e02833bd8ed70e93deac60a14b5c66d31c6eac65962bf54ea"}
Oct 30 12:56:32.102450 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.102411    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz" event={"ID":"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56","Type":"ContainerStarted","Data":"1c4544afd27f657628273ca69886f70932c2fa5a4ba34123ba5e5ce43203e091"}
Oct 30 12:56:32.103735 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.103677    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdmv9" event={"ID":"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9","Type":"ContainerStarted","Data":"9a33816ef2cd30b219a540c24f03ef7342a59fb9412e1901e2c6f606ab997294"}
Oct 30 12:56:32.123856 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.123787    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c56xj" event={"ID":"39289142-2079-4b45-ba09-8e06aed12276","Type":"ContainerStarted","Data":"e2439f6a602acc4de55b7b91cc20a90f0b8793364c7875738b90a8335828cecd"}
Oct 30 12:56:32.125555 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.125493    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97p84" event={"ID":"f4471adf-01e8-47ae-8056-c670541a735c","Type":"ContainerStarted","Data":"0f5ddbb728fbacf85b52607447cfa8da1ebc445a470d9cb53306a0bf051f5009"}
Oct 30 12:56:32.127386 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.127306    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t7zb4" event={"ID":"2926b50e-e07d-445d-bbec-eab0005a59bf","Type":"ContainerStarted","Data":"ce1ac0a68e3b02b8a5bec0ac63626c10ae12ae696f33bb32fd09b114e0270fbd"}
Oct 30 12:56:32.130375 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.130242    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5gn6d" event={"ID":"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d","Type":"ContainerStarted","Data":"ea7df2b853a22f5e1cdcfdf57a42f434621e5343f5be7d7cb43ecc05971a6d79"}
Oct 30 12:56:32.133775 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.133748    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k6rf5" event={"ID":"0adce241-e2df-40ec-8976-341e00ebc074","Type":"ContainerStarted","Data":"71446ba892b17259700ac5db04ae23a27f073e8569b9bada878a8427adf79bc9"}
Oct 30 12:56:32.139875 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.139798    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerStarted","Data":"a1ca5baef5c7169c7da2ba7c009c908f20eca9358755d105faf0641fcf7f5cfd"}
Oct 30 12:56:32.148057 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.148032    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rlghf" event={"ID":"853421d2-ef94-4cbd-b251-1f4d82bfb45e","Type":"ContainerStarted","Data":"7b63fe8b9bff669d6d61a0846f7fa93f8ac602b5b3f2f2b049b537fa5b2efbf2"}
Oct 30 12:56:32.157827 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.157803    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal" event={"ID":"dbd004dde5b9ed65a173b78115c079e0","Type":"ContainerStarted","Data":"205ac379ad661cb070f4d3c8c022adf712cde46fa98d5a1ecd97ddcb92117ccc"}
Oct 30 12:56:32.676857 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.676746    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:32.676857 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.676828    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:32.677069 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:32.676954    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:32.677069 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:32.677013    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs podName:1062c6b4-6828-4e92-92a5-c78eccb1fe26 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:34.676996532 +0000 UTC m=+6.126073138 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs") pod "network-metrics-daemon-xtz6h" (UID: "1062c6b4-6828-4e92-92a5-c78eccb1fe26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:32.677420 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:32.677402    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:32.677496 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:32.677456    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret podName:c69d9666-d791-48fe-aad3-7a91f2ab2d2c nodeName:}" failed. No retries permitted until 2026-10-30 12:56:34.677441738 +0000 UTC m=+6.126518341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret") pod "global-pull-secret-syncer-gkrqb" (UID: "c69d9666-d791-48fe-aad3-7a91f2ab2d2c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:32.777650 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:32.777609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:32.777817 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:32.777783    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:32.777817 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:32.777809    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:32.777935 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:32.777823    2583 projected.go:196] Error preparing data for projected volume kube-api-access-srms2 for pod openshift-network-diagnostics/network-check-target-xc6hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:32.777935 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:32.777875    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2 podName:4faa20e7-dd05-4416-9b9e-ed52b91d388d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:34.777857299 +0000 UTC m=+6.226933904 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-srms2" (UniqueName: "kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2") pod "network-check-target-xc6hk" (UID: "4faa20e7-dd05-4416-9b9e-ed52b91d388d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:33.013159 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:33.013054    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:51:30 +0000 UTC" deadline="2028-08-12 22:18:24.971442212 +0000 UTC"
Oct 30 12:56:33.013159 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:33.013091    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15657h21m51.958355923s"
Oct 30 12:56:33.086957 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:33.086926    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:33.087111 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:33.087053    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:33.087438 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:33.087417    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:33.087529 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:33.087512    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:33.087591 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:33.087579    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:33.087682 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:33.087660    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:33.180602 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:33.180568    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal" event={"ID":"9b3f18c352b4e52d3d8a287983d93c2b","Type":"ContainerStarted","Data":"b538f5c85104384a835e754e6fa585c049abcf5ef8076f553365c8704390d433"}
Oct 30 12:56:33.223032 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:33.222975    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal" podStartSLOduration=3.222963774 podStartE2EDuration="3.222963774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:33.222803666 +0000 UTC m=+4.671880276" watchObservedRunningTime="2026-10-30 12:56:33.222963774 +0000 UTC m=+4.672040384"
Oct 30 12:56:33.223517 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:33.223488    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal" podStartSLOduration=3.223479877 podStartE2EDuration="3.223479877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:32.200092784 +0000 UTC m=+3.649169393" watchObservedRunningTime="2026-10-30 12:56:33.223479877 +0000 UTC m=+4.672556488"
Oct 30 12:56:34.693889 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:34.693803    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:34.695113 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:34.693891    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:34.695113 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:34.694024    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:34.695113 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:34.694085    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs podName:1062c6b4-6828-4e92-92a5-c78eccb1fe26 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:38.694065191 +0000 UTC m=+10.143141796 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs") pod "network-metrics-daemon-xtz6h" (UID: "1062c6b4-6828-4e92-92a5-c78eccb1fe26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:34.695113 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:34.694494    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:34.695113 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:34.694543    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret podName:c69d9666-d791-48fe-aad3-7a91f2ab2d2c nodeName:}" failed. No retries permitted until 2026-10-30 12:56:38.694529852 +0000 UTC m=+10.143606445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret") pod "global-pull-secret-syncer-gkrqb" (UID: "c69d9666-d791-48fe-aad3-7a91f2ab2d2c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:34.794738 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:34.794708    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:34.794913 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:34.794861    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:34.794913 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:34.794882    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:34.794913 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:34.794891    2583 projected.go:196] Error preparing data for projected volume kube-api-access-srms2 for pod openshift-network-diagnostics/network-check-target-xc6hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:34.795078 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:34.794938    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2 podName:4faa20e7-dd05-4416-9b9e-ed52b91d388d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:38.794923702 +0000 UTC m=+10.244000289 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-srms2" (UniqueName: "kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2") pod "network-check-target-xc6hk" (UID: "4faa20e7-dd05-4416-9b9e-ed52b91d388d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:35.087839 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:35.087762    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:35.087996 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:35.087890    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:35.088273 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:35.088253    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:35.088405 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:35.088382    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:35.088472 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:35.088442    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:35.088575 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:35.088555    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:35.194371 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:35.194337    2583 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="8112e4ae72f183aa07e50045111c4f206b3cb740823e9c871a5c9a80780f3056" exitCode=0
Oct 30 12:56:35.194543 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:35.194395    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97p84" event={"ID":"f4471adf-01e8-47ae-8056-c670541a735c","Type":"ContainerDied","Data":"8112e4ae72f183aa07e50045111c4f206b3cb740823e9c871a5c9a80780f3056"}
Oct 30 12:56:36.200333 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:36.200273    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c56xj" event={"ID":"39289142-2079-4b45-ba09-8e06aed12276","Type":"ContainerStarted","Data":"9ec4b2a8975d67dd25f312bde1269c998ec9bf3e9619b6cac5c6ed224b2b8472"}
Oct 30 12:56:36.393303 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:36.393222    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:36.393831 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:36.393812    2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:36.437688 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:36.437621    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c56xj" podStartSLOduration=3.536223623 podStartE2EDuration="7.437601025s" totalImagesPullingTime="3.901377402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:36.244125902 +0000 UTC m=+7.693202515" watchObservedRunningTime="2026-10-30 12:56:36.437601025 +0000 UTC m=+7.886677636"
Oct 30 12:56:37.086522 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:37.086452    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:37.086659 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:37.086452    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:37.086659 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:37.086606    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:37.086659 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:37.086607    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:37.086752 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:37.086694    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:37.086786 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:37.086771    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:37.205322 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:37.205203    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t7zb4" event={"ID":"2926b50e-e07d-445d-bbec-eab0005a59bf","Type":"ContainerStarted","Data":"03b9be60a4ec1eac89fd3bf340d7df3c788b0a9abff1b5be60ae3e37c8f9f1c3"}
Oct 30 12:56:37.208787 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:37.208758    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz" event={"ID":"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56","Type":"ContainerStarted","Data":"70fa49fd990551af1f93e64dfc03ba56cba67317962c20eeb241dd73a7f19dca"}
Oct 30 12:56:38.213760 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:38.213322    2583 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 12:56:38.300496 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:38.300452    2583 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 12:56:38.731158 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:38.731116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:38.731350 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:38.731215    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:38.731432 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:38.731364    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:38.731432 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:38.731429    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs podName:1062c6b4-6828-4e92-92a5-c78eccb1fe26 nodeName:}" failed. No retries permitted until 2026-10-30 12:56:46.73141019 +0000 UTC m=+18.180486783 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs") pod "network-metrics-daemon-xtz6h" (UID: "1062c6b4-6828-4e92-92a5-c78eccb1fe26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:38.731838 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:38.731814    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:38.731943 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:38.731873    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret podName:c69d9666-d791-48fe-aad3-7a91f2ab2d2c nodeName:}" failed. No retries permitted until 2026-10-30 12:56:46.731857153 +0000 UTC m=+18.180933758 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret") pod "global-pull-secret-syncer-gkrqb" (UID: "c69d9666-d791-48fe-aad3-7a91f2ab2d2c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:38.832138 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:38.832106    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:38.832325 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:38.832227    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:38.832325 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:38.832243    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:38.832325 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:38.832252    2583 projected.go:196] Error preparing data for projected volume kube-api-access-srms2 for pod openshift-network-diagnostics/network-check-target-xc6hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:38.832325 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:38.832315    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2 podName:4faa20e7-dd05-4416-9b9e-ed52b91d388d nodeName:}" failed. No retries permitted until 2026-10-30 12:56:46.832298326 +0000 UTC m=+18.281374928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-srms2" (UniqueName: "kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2") pod "network-check-target-xc6hk" (UID: "4faa20e7-dd05-4416-9b9e-ed52b91d388d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:39.007150 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:39.007063    2583 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T12:56:38.300477714Z","UUID":"520f7d54-b7e8-49ed-af8c-76d42d68543f","Handler":null,"Name":"","Endpoint":""}
Oct 30 12:56:39.009579 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:39.009047    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 12:56:39.009579 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:39.009079    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 12:56:39.087409 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:39.086852    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:39.087409 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:39.086961    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:39.087409 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:39.087038    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:39.087409 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:39.087121    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:39.087409 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:39.087172    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:39.087409 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:39.087332    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:39.219340 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:39.219281    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz" event={"ID":"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56","Type":"ContainerStarted","Data":"9425869d52bce5e9786dbcdb61bb7f1e92fd06c66437a3f7bf77a7b178743f48"}
Oct 30 12:56:39.221748 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:39.221675    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k6rf5" event={"ID":"0adce241-e2df-40ec-8976-341e00ebc074","Type":"ContainerStarted","Data":"7d9705dc3b343644a6c7c9f5344824ded352e54918c6348862578d8ea5a62fce"}
Oct 30 12:56:39.268247 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:39.268152    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t7zb4" podStartSLOduration=5.470995717 podStartE2EDuration="10.268137947s" totalImagesPullingTime="4.79714223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:37.283323748 +0000 UTC m=+8.732400359" watchObservedRunningTime="2026-10-30 12:56:39.268137947 +0000 UTC m=+10.717214558"
Oct 30 12:56:40.226697 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:40.226198    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rlghf" event={"ID":"853421d2-ef94-4cbd-b251-1f4d82bfb45e","Type":"ContainerStarted","Data":"5387395efb060ebbe5b0c5991f4850978c9cd733c7e7d65651d0ab3f90223d19"}
Oct 30 12:56:40.229176 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:40.229146    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz" event={"ID":"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56","Type":"ContainerStarted","Data":"47babf5417849e8a0441c1050ddb37979dd9a8a152a2822bcb98fe8c6c527463"}
Oct 30 12:56:40.232714 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:40.232684    2583 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="8c97bddc001d013ac39da9299c0dacbedcb82d2440c3e0318ab9ade053328a30" exitCode=0
Oct 30 12:56:40.233770 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:40.233748    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97p84" event={"ID":"f4471adf-01e8-47ae-8056-c670541a735c","Type":"ContainerDied","Data":"8c97bddc001d013ac39da9299c0dacbedcb82d2440c3e0318ab9ade053328a30"}
Oct 30 12:56:40.273155 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:40.270499    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-rlghf" podStartSLOduration=3.522555073 podStartE2EDuration="11.270485441s" totalImagesPullingTime="7.747930368s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:40.26901088 +0000 UTC m=+11.718087467" watchObservedRunningTime="2026-10-30 12:56:40.270485441 +0000 UTC m=+11.719562048"
Oct 30 12:56:40.273155 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:40.270869    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-k6rf5" podStartSLOduration=3.716643345 podStartE2EDuration="11.270857867s" totalImagesPullingTime="7.554214522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:39.268503213 +0000 UTC m=+10.717579823" watchObservedRunningTime="2026-10-30 12:56:40.270857867 +0000 UTC m=+11.719934479"
Oct 30 12:56:40.314747 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:40.314682    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz" podStartSLOduration=3.4603904 podStartE2EDuration="11.314663439s" totalImagesPullingTime="7.854273039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:40.31400042 +0000 UTC m=+11.763077031" watchObservedRunningTime="2026-10-30 12:56:40.314663439 +0000 UTC m=+11.763740056"
Oct 30 12:56:41.085858 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:41.085799    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:41.086026 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:41.085880    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:41.086026 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:41.086007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:41.086128 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:41.086022    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:41.086128 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:41.086112    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:41.086231 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:41.086204    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:41.237078 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:41.236996    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sdmv9" event={"ID":"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9","Type":"ContainerStarted","Data":"233b8a6971eaa7517cac7882118cfaff44e65bd12b42f7c4e2efc01fef2b2949"}
Oct 30 12:56:41.239359 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:41.239268    2583 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="5ccd197fa405996a479011b4194a7aa634d29b979603cbe064b12bc13fcbf9ee" exitCode=0
Oct 30 12:56:41.239359 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:41.239323    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97p84" event={"ID":"f4471adf-01e8-47ae-8056-c670541a735c","Type":"ContainerDied","Data":"5ccd197fa405996a479011b4194a7aa634d29b979603cbe064b12bc13fcbf9ee"}
Oct 30 12:56:41.241134 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:41.241090    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5gn6d" event={"ID":"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d","Type":"ContainerStarted","Data":"c71200ab32973de8bd725371c336fab674378e5253112a8ca3d054b460545ff2"}
Oct 30 12:56:41.278475 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:41.278429    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sdmv9" podStartSLOduration=4.6561841489999996 podStartE2EDuration="12.278416307s" totalImagesPullingTime="7.622232158s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:41.278110639 +0000 UTC m=+12.727187249" watchObservedRunningTime="2026-10-30 12:56:41.278416307 +0000 UTC m=+12.727492917"
Oct 30 12:56:41.322395 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:41.322337    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5gn6d" podStartSLOduration=2.719347821 podStartE2EDuration="12.322319877s" totalImagesPullingTime="9.602972056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:41.321859392 +0000 UTC m=+12.770936014" watchObservedRunningTime="2026-10-30 12:56:41.322319877 +0000 UTC m=+12.771396483"
Oct 30 12:56:42.247240 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:42.246959    2583 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="6b242cff777197d1224eeb50c91a017629c67077b1e1415d3ffecac07a13fb74" exitCode=0
Oct 30 12:56:42.247240 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:42.247099    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97p84" event={"ID":"f4471adf-01e8-47ae-8056-c670541a735c","Type":"ContainerDied","Data":"6b242cff777197d1224eeb50c91a017629c67077b1e1415d3ffecac07a13fb74"}
Oct 30 12:56:43.085837 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:43.085803    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:43.085999 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:43.085925    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:43.086382 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:43.086356    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:43.086553 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:43.086526    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:43.086627 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:43.086589    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:43.086686 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:43.086666    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:44.254408 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:44.254199    2583 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="ae248779d1c2d6205f2830472f027dd9a5de172c1e3cfcf8775607d9ec4dcded" exitCode=0
Oct 30 12:56:44.254408 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:44.254303    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97p84" event={"ID":"f4471adf-01e8-47ae-8056-c670541a735c","Type":"ContainerDied","Data":"ae248779d1c2d6205f2830472f027dd9a5de172c1e3cfcf8775607d9ec4dcded"}
Oct 30 12:56:45.085351 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:45.085315    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:45.085557 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:45.085320    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:45.085557 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:45.085462    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:45.085645 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:45.085558    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:45.085645 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:45.085322    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:45.085724 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:45.085646    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:45.259724 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:45.259696    2583 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="0e3012534e87b6e45f60966061605ac0050f5cf6baefaeccfe18bedfcccb05c3" exitCode=0
Oct 30 12:56:45.260121 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:45.259746    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97p84" event={"ID":"f4471adf-01e8-47ae-8056-c670541a735c","Type":"ContainerDied","Data":"0e3012534e87b6e45f60966061605ac0050f5cf6baefaeccfe18bedfcccb05c3"}
Oct 30 12:56:46.266003 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.265773    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97p84" event={"ID":"f4471adf-01e8-47ae-8056-c670541a735c","Type":"ContainerStarted","Data":"2697e7f77806155cbc371452084d62a2b84911faefe2ca4d563cde4f612366da"}
Oct 30 12:56:46.268370 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.268350    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 12:56:46.268688 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.268670    2583 generic.go:356] "Generic (PLEG): container finished" podID="162db930-0919-4366-ac55-b8e647cb714b" containerID="15c51bdd0d9fa7a74f73c940004eb7c1c0f5d5888e9e51fef64d7e2da121ba14" exitCode=1
Oct 30 12:56:46.268744 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.268702    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerStarted","Data":"114de86311082489c34f0311ee21da02ff1561c1e75cbf0e94e448d14ae88ff0"}
Oct 30 12:56:46.268776 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.268750    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerStarted","Data":"31900349b8d57f9c6fa3a07ed3f9ee4cc10ab6b206a0c6edc79af44f6f277368"}
Oct 30 12:56:46.268806 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.268780    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerStarted","Data":"1c4cb8af8015183bd0cfc36b3c3c97cc9ac93ff9871fa79a99dcd2407c11aa7e"}
Oct 30 12:56:46.268844 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.268808    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerStarted","Data":"92d81c1121f3df28dad2b977d7ddbe59cd20ffc71aaf581ec717b872090c6bc8"}
Oct 30 12:56:46.268844 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.268835    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerDied","Data":"15c51bdd0d9fa7a74f73c940004eb7c1c0f5d5888e9e51fef64d7e2da121ba14"}
Oct 30 12:56:46.268910 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.268862    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerStarted","Data":"f8a2c671246ac366bd9b3df24f41c873e8b1d77f5a9269a63930f3798fca39b8"}
Oct 30 12:56:46.316803 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.316758    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-97p84" podStartSLOduration=7.280407219 podStartE2EDuration="17.316746733s" totalImagesPullingTime="10.036339514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:46.316402465 +0000 UTC m=+17.765479074" watchObservedRunningTime="2026-10-30 12:56:46.316746733 +0000 UTC m=+17.765823341"
Oct 30 12:56:46.795213 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.795190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:46.795368 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.795230    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:46.795368 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:46.795357    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:46.795453 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:46.795411    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret podName:c69d9666-d791-48fe-aad3-7a91f2ab2d2c nodeName:}" failed. No retries permitted until 2026-10-30 12:57:02.795397435 +0000 UTC m=+34.244474022 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret") pod "global-pull-secret-syncer-gkrqb" (UID: "c69d9666-d791-48fe-aad3-7a91f2ab2d2c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:56:46.795453 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:46.795357    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:46.795514 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:46.795477    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs podName:1062c6b4-6828-4e92-92a5-c78eccb1fe26 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:02.795466525 +0000 UTC m=+34.244543128 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs") pod "network-metrics-daemon-xtz6h" (UID: "1062c6b4-6828-4e92-92a5-c78eccb1fe26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:56:46.895741 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:46.895717    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:46.895862 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:46.895849    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:56:46.895910 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:46.895866    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:56:46.895910 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:46.895876    2583 projected.go:196] Error preparing data for projected volume kube-api-access-srms2 for pod openshift-network-diagnostics/network-check-target-xc6hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:46.895967 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:46.895918    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2 podName:4faa20e7-dd05-4416-9b9e-ed52b91d388d nodeName:}" failed. No retries permitted until 2026-10-30 12:57:02.895905933 +0000 UTC m=+34.344982533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-srms2" (UniqueName: "kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2") pod "network-check-target-xc6hk" (UID: "4faa20e7-dd05-4416-9b9e-ed52b91d388d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:56:47.085332 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:47.085246    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:47.085332 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:47.085270    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:47.085485 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:47.085379    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:47.085485 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:47.085405    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:47.085485 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:47.085475    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:47.085576 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:47.085548    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:48.275324 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:48.275283    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 12:56:48.275791 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:48.275650    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerStarted","Data":"4a5ffce09a4eb17fb3de33f282e75ed1c96d82f5cd78bbe45666c882c90bfb9f"}
Oct 30 12:56:49.087219 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:49.087187    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:49.087379 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:49.087309    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:49.087379 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:49.087320    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:49.087379 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:49.087333    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:49.087482 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:49.087395    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:49.087482 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:49.087451    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:50.283089 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:50.283068    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 12:56:50.283745 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:50.283437    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerStarted","Data":"4ca4f225b3cb8131b0a25a6754d3b2d97667f4bc769bf3f0c0dfe24666a2a77c"}
Oct 30 12:56:50.284481 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:50.284453    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:50.284603 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:50.284574    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:50.285117 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:50.285098    2583 scope.go:122] "RemoveContainer" containerID="15c51bdd0d9fa7a74f73c940004eb7c1c0f5d5888e9e51fef64d7e2da121ba14"
Oct 30 12:56:50.303615 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:50.303589    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:51.085667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.085479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:51.085868 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.085479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:51.085868 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:51.085759    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:51.085868 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.085525    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:51.085868 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:51.085846    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:51.086054 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:51.085889    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:51.289156 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.289129    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 12:56:51.289580 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.289464    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" event={"ID":"162db930-0919-4366-ac55-b8e647cb714b","Type":"ContainerStarted","Data":"0bee28d2f4a76636223bfb0c66cfc77428b94efcbccd1adb4b0f564ea08c320d"}
Oct 30 12:56:51.289714 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.289695    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:51.304455 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.304432    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:56:51.386767 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.386685    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p" podStartSLOduration=8.04307159 podStartE2EDuration="22.38667362s" totalImagesPullingTime="14.34360203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:51.372898162 +0000 UTC m=+22.821974771" watchObservedRunningTime="2026-10-30 12:56:51.38667362 +0000 UTC m=+22.835750231"
Oct 30 12:56:51.522821 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.522788    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:51.522950 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.522916    2583 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 12:56:51.523455 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:51.523432    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:56:53.049662 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:53.049628    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:56:53.085621 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:53.085590    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:53.085621 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:53.085607    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:53.085801 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:53.085719    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:53.085866 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:53.085841    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:53.085928 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:53.085915    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:53.086010 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:53.085993    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:53.273621 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:53.273590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:56:54.311517 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:54.311487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:56:54.477229 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:54.477197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:56:54.717098 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:54.717031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:56:55.085396 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:55.085304    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:55.085538 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:55.085403    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:55.085538 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:55.085420    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:55.085538 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:55.085431    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:55.085538 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:55.085469    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:55.085677 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:55.085558    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:56.466515 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:56.466473    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:56:56.616144 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:56.616111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:56:56.770016 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:56.769785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:56:57.085695 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:57.085605    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:57.085695 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:57.085623    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:57.085937 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:57.085726    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:57.085937 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:57.085761    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:57.085937 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:57.085830    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:56:57.085937 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:57.085908    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:59.086050 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:59.086019    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:56:59.086438 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:59.086097    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:56:59.086438 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:59.086190    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:56:59.086438 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:56:59.086319    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:56:59.086438 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:59.086397    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:56:59.086438 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:56:59.086403    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:00.021955 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:00.021899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:00.247536 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:00.247505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:00.550257 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:00.550226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:00.743210 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:00.743184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:00.917503 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:00.917433    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:01.085545 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:01.085518    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:01.085545 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:01.085545    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:01.085727 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:01.085676    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:01.085727 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:01.085722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:01.085807 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:01.085791    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:01.085870 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:01.085855    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:02.001801 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.001768    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:02.382510 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.382439    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xc6hk"]
Oct 30 12:57:02.382653 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.382560    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:02.382703 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.382664    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:02.386799 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.385805    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gkrqb"]
Oct 30 12:57:02.386799 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.385897    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:02.386799 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.385969    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:02.386799 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.386652    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xtz6h"]
Oct 30 12:57:02.386799 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.386711    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:02.386799 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.386772    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:02.818873 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.818790    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:02.818873 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.818831    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:02.819082 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.818928    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:57:02.819082 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.818936    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:57:02.819082 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.818977    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret podName:c69d9666-d791-48fe-aad3-7a91f2ab2d2c nodeName:}" failed. No retries permitted until 2026-10-30 12:57:34.818964323 +0000 UTC m=+66.268040911 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret") pod "global-pull-secret-syncer-gkrqb" (UID: "c69d9666-d791-48fe-aad3-7a91f2ab2d2c") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:57:02.819082 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.818990    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs podName:1062c6b4-6828-4e92-92a5-c78eccb1fe26 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:34.81898378 +0000 UTC m=+66.268060368 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs") pod "network-metrics-daemon-xtz6h" (UID: "1062c6b4-6828-4e92-92a5-c78eccb1fe26") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:57:02.919944 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:02.919911    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:02.920098 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.920064    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:57:02.920098 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.920087    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:57:02.920098 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.920096    2583 projected.go:196] Error preparing data for projected volume kube-api-access-srms2 for pod openshift-network-diagnostics/network-check-target-xc6hk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:57:02.920217 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:02.920144    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2 podName:4faa20e7-dd05-4416-9b9e-ed52b91d388d nodeName:}" failed. No retries permitted until 2026-10-30 12:57:34.920131207 +0000 UTC m=+66.369207795 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-srms2" (UniqueName: "kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2") pod "network-check-target-xc6hk" (UID: "4faa20e7-dd05-4416-9b9e-ed52b91d388d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:57:04.085332 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:04.085275    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:04.085332 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:04.085329    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:04.085712 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:04.085278    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:04.085712 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:04.085404    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:04.085712 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:04.085481    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:04.085712 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:04.085560    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:04.530174 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:04.530146    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:05.411337 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:05.411307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:05.567618 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:05.567586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:06.084670 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:06.084641    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:06.084855 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:06.084749    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:06.084855 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:06.084764    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:06.084855 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:06.084774    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:06.084855 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:06.084838    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:06.085066 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:06.084895    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:06.323030 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:06.323001    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:06.665953 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:06.665924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:06.838302 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:06.838215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:07.023529 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:07.023498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:07.256341 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:07.256266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:07.417501 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:07.417328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:08.085503 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:08.085476    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:08.085503 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:08.085504    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:08.085905 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:08.085579    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:08.085905 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:08.085576    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:08.085905 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:08.085667    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:08.085905 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:08.085712    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:08.622062 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:08.622033    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:08.771697 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:08.771666    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:08.911135 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:08.911063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:09.053709 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:09.053678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:09.229745 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:09.229681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:09.418398 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:09.418362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:10.009797 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:10.009770    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rlghf_853421d2-ef94-4cbd-b251-1f4d82bfb45e/dns-node-resolver/0.log"
Oct 30 12:57:10.084640 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:10.084609    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:10.084804 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:10.084682    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:10.084804 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:10.084709    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:10.084915 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:10.084824    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:10.084915 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:10.084861    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:10.085013 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:10.084935    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:10.347452 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:10.347382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:10.619538 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:10.619465    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:10.805707 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:10.805685    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t7zb4_2926b50e-e07d-445d-bbec-eab0005a59bf/node-ca/0.log"
Oct 30 12:57:10.825300 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:10.825254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:11.019500 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:11.019465    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:11.161894 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:11.161861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:11.363200 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:11.363118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:11.511659 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:11.511629    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:12.084804 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:12.084777    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:12.084930 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:12.084842    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:12.084930 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:12.084868    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:12.084998 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:12.084945    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:12.084998 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:12.084981    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:12.085075 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:12.085059    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:12.647216 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:12.647188    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:12.806651 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:12.806623    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:12.964861 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:12.964832    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:13.123334 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:13.123299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:13.277013 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:13.276926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:13.474054 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:13.474024    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:14.008536 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:14.008507    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:14.085026 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:14.084997    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:14.085172 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:14.085091    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:14.085172 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:14.085096    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:14.085172 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:14.085094    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:14.085276 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:14.085172    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:14.085276 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:14.085262    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:14.188461 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:14.188433    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:14.340889 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:14.340814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:14.647241 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:14.647166    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:14.830408 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:14.830377    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:14.997952 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:14.997926    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:15.145950 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:15.145918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:15.303561 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:15.303488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:15.456633 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:15.456606    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:16.012963 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:16.012935    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:16.085570 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:16.085543    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:16.085695 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:16.085605    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:16.085695 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:16.085629    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:16.085770 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:16.085728    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:16.085808 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:16.085765    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:16.085845 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:16.085831    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:16.219794 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:16.219754    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:18.085078 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:18.085049    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:18.085502 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:18.085162    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:18.085502 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:18.085168    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:18.085502 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:18.085245    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:18.085502 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:18.085248    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:18.085502 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:18.085355    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:19.310083 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:19.310052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:19.538572 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:19.538542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:19.695712 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:19.695686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:20.085614 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:20.085538    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:20.085614 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:20.085564    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:20.085791 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:20.085655    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:20.085791 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:20.085706    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:20.085791 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:20.085780    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:20.085881 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:20.085853    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:21.528821 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:21.528785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:21.675561 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:21.675534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:22.084826 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:22.084796    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:22.084826 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:22.084823    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:22.085036 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:22.084897    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:22.085036 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:22.084928    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:22.085036 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:22.084956    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:22.085036 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:22.085009    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:23.304494 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:23.304472    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:24.085340 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:24.085278    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:24.085499 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:24.085409    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:24.085499 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:24.085314    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:24.085499 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:24.085457    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:24.085640 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:24.085536    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:24.085640 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:24.085620    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:24.120088 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:24.120062    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:25.802756 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:25.802722    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:25.972624 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:25.972588    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:26.085729 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.085647    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:26.085889 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.085648    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:26.085889 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:26.085767    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtz6h" podUID="1062c6b4-6828-4e92-92a5-c78eccb1fe26"
Oct 30 12:57:26.085889 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.085648    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:26.086060 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:26.085828    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xc6hk" podUID="4faa20e7-dd05-4416-9b9e-ed52b91d388d"
Oct 30 12:57:26.086060 ip-10-0-130-180 kubenswrapper[2583]: E1030 12:57:26.085922    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gkrqb" podUID="c69d9666-d791-48fe-aad3-7a91f2ab2d2c"
Oct 30 12:57:26.157505 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.157474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:26.473476 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.473455    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:26.640826 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.640791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:26.845862 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.845793    2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeReady"
Oct 30 12:57:26.846200 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.845902    2583 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 12:57:26.913376 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.913343    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fqj4w"]
Oct 30 12:57:26.916215 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.916199    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:26.924533 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.924509    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gphtb"]
Oct 30 12:57:26.927983 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.927968    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:26.933260 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.933246    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 12:57:26.933412 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.933396    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5bbfv\""
Oct 30 12:57:26.933761 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.933749    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 12:57:26.939075 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.939057    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qlp5s"]
Oct 30 12:57:26.941838 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.941816    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:26.942845 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.942625    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fqj4w"]
Oct 30 12:57:26.953785 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.953767    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gphtb"]
Oct 30 12:57:26.980769 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.980751    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qlp5s"]
Oct 30 12:57:26.990205 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990181    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-crio-socket\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:26.990301 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990210    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:26.990301 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990244    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/793c2518-45fa-46e0-9a23-766ae40d922b-metrics-tls\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:26.990378 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990308    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:26.990411 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990381    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/793c2518-45fa-46e0-9a23-766ae40d922b-tmp-dir\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:26.990411 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990402    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-cert\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:26.990473 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990416    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87s78\" (UniqueName: \"kubernetes.io/projected/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-kube-api-access-87s78\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:26.990473 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990435    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xhvx\" (UniqueName: \"kubernetes.io/projected/793c2518-45fa-46e0-9a23-766ae40d922b-kube-api-access-7xhvx\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:26.990558 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990544    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/793c2518-45fa-46e0-9a23-766ae40d922b-config-volume\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:26.990590 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990565    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-data-volume\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:26.990621 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:26.990613    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74s7t\" (UniqueName: \"kubernetes.io/projected/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-api-access-74s7t\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.033087 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.033067    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 12:57:27.033087 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.033066    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 12:57:27.033087 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.033066    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kmvh7\""
Oct 30 12:57:27.033263 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.033111    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-sdfnl\""
Oct 30 12:57:27.033598 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.033577    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 12:57:27.033674 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.033605    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 12:57:27.033674 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.033627    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 12:57:27.033742 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.033679    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 12:57:27.033835 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.033820    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 12:57:27.091816 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.091792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-crio-socket\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.091921 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.091821    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.091921 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.091861    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/793c2518-45fa-46e0-9a23-766ae40d922b-metrics-tls\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:27.091921 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.091885    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.091921 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.091909    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/793c2518-45fa-46e0-9a23-766ae40d922b-tmp-dir\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:27.092091 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.091930    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-cert\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:27.092091 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.091951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87s78\" (UniqueName: \"kubernetes.io/projected/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-kube-api-access-87s78\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:27.092091 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.092021    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-crio-socket\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.092091 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.092023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xhvx\" (UniqueName: \"kubernetes.io/projected/793c2518-45fa-46e0-9a23-766ae40d922b-kube-api-access-7xhvx\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:27.092279 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.092102    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/793c2518-45fa-46e0-9a23-766ae40d922b-config-volume\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:27.092279 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.092133    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-data-volume\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.092279 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.092159    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74s7t\" (UniqueName: \"kubernetes.io/projected/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-api-access-74s7t\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.092448 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.092358    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/793c2518-45fa-46e0-9a23-766ae40d922b-tmp-dir\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:27.092621 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.092595    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-data-volume\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.092734 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.092604    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.092734 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.092701    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/793c2518-45fa-46e0-9a23-766ae40d922b-config-volume\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:27.095667 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.095642    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.095769 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.095684    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/793c2518-45fa-46e0-9a23-766ae40d922b-metrics-tls\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:27.095769 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.095718    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-cert\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:27.123902 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.123855    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xhvx\" (UniqueName: \"kubernetes.io/projected/793c2518-45fa-46e0-9a23-766ae40d922b-kube-api-access-7xhvx\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:27.125160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.125137    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74s7t\" (UniqueName: \"kubernetes.io/projected/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-api-access-74s7t\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.125785 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.125764    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87s78\" (UniqueName: \"kubernetes.io/projected/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-kube-api-access-87s78\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:27.148925 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.148901    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:27.230230 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.230198    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:27.237481 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.237459    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:27.256495 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.256473    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:27.302469 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.300554    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:27.464889 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.464864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:27.573551 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.573349    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fqj4w"]
Oct 30 12:57:27.576183 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.576158    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gphtb"]
Oct 30 12:57:27.576480 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:27.576459    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793c2518_45fa_46e0_9a23_766ae40d922b.slice/crio-60fa67f9cce7dfdda00d96d1f2f07818fb787f1d8e8951492ba1b524941495ef WatchSource:0}: Error finding container 60fa67f9cce7dfdda00d96d1f2f07818fb787f1d8e8951492ba1b524941495ef: Status 404 returned error can't find the container with id 60fa67f9cce7dfdda00d96d1f2f07818fb787f1d8e8951492ba1b524941495ef
Oct 30 12:57:27.581618 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:27.581593    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c3d5df_2428_490f_925a_f67ef2e8c0b6.slice/crio-8321f08e19f4910820e6872cfb6f88739d17c098e9875cfa591d843b3995946c WatchSource:0}: Error finding container 8321f08e19f4910820e6872cfb6f88739d17c098e9875cfa591d843b3995946c: Status 404 returned error can't find the container with id 8321f08e19f4910820e6872cfb6f88739d17c098e9875cfa591d843b3995946c
Oct 30 12:57:27.617963 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:27.617935    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qlp5s"]
Oct 30 12:57:27.620755 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:27.620723    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a9daaf5_4ea6_472e_9972_68cfeee5ae16.slice/crio-4f0d02a497aea0a09e35155819127d1154dc15d08de78b7cf3be97c45a3ae79c WatchSource:0}: Error finding container 4f0d02a497aea0a09e35155819127d1154dc15d08de78b7cf3be97c45a3ae79c: Status 404 returned error can't find the container with id 4f0d02a497aea0a09e35155819127d1154dc15d08de78b7cf3be97c45a3ae79c
Oct 30 12:57:28.087349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.084991    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:28.087349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.085658    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:28.087349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.086074    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:28.103503 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.103478    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 12:57:28.103937 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.103914    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v4tv9\""
Oct 30 12:57:28.104033 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.103973    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vg7bt\""
Oct 30 12:57:28.104095 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.104058    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-sqtw6\""
Oct 30 12:57:28.104095 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.104064    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 12:57:28.104196 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.104125    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 12:57:28.104280 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.104262    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 12:57:28.322218 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.322196    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rlghf_853421d2-ef94-4cbd-b251-1f4d82bfb45e/dns-node-resolver/0.log"
Oct 30 12:57:28.373706 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.373672    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qlp5s" event={"ID":"4a9daaf5-4ea6-472e-9972-68cfeee5ae16","Type":"ContainerStarted","Data":"46592fac28dced2718d53ce03b1648f5afc4410a60576b927c5a677ad859f5d3"}
Oct 30 12:57:28.373858 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.373740    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qlp5s" event={"ID":"4a9daaf5-4ea6-472e-9972-68cfeee5ae16","Type":"ContainerStarted","Data":"25499d3947520ae4bd1b43a7059bc2aae290bb9b6a1fc962401932c97a839384"}
Oct 30 12:57:28.373858 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.373766    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qlp5s" event={"ID":"4a9daaf5-4ea6-472e-9972-68cfeee5ae16","Type":"ContainerStarted","Data":"4f0d02a497aea0a09e35155819127d1154dc15d08de78b7cf3be97c45a3ae79c"}
Oct 30 12:57:28.375091 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.375066    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gphtb" event={"ID":"b5c3d5df-2428-490f-925a-f67ef2e8c0b6","Type":"ContainerStarted","Data":"8321f08e19f4910820e6872cfb6f88739d17c098e9875cfa591d843b3995946c"}
Oct 30 12:57:28.378520 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:28.378497    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqj4w" event={"ID":"793c2518-45fa-46e0-9a23-766ae40d922b","Type":"ContainerStarted","Data":"60fa67f9cce7dfdda00d96d1f2f07818fb787f1d8e8951492ba1b524941495ef"}
Oct 30 12:57:29.135717 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.135682    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:29.220612 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.220577    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"]
Oct 30 12:57:29.224818 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.224796    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.241612 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.241586    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 12:57:29.241727 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.241719    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 12:57:29.241858 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.241845    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 12:57:29.241950 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.241937    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 12:57:29.242015 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.242004    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2qk6c\""
Oct 30 12:57:29.242162 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.242147    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 12:57:29.247689 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.247661    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"]
Oct 30 12:57:29.301173 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.301128    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:29.314013 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.313177    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.314013 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.313243    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.314013 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.313277    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpjcn\" (UniqueName: \"kubernetes.io/projected/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-kube-api-access-vpjcn\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.314013 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.313338    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.321569 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.321540    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t7zb4_2926b50e-e07d-445d-bbec-eab0005a59bf/node-ca/0.log"
Oct 30 12:57:29.413987 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.413901    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.413987 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.413954    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.414248 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.413988    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpjcn\" (UniqueName: \"kubernetes.io/projected/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-kube-api-access-vpjcn\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.414248 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.414033    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.414860 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.414829    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.417101 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.417079    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.421787 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.421763    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.439983 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.439948    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpjcn\" (UniqueName: \"kubernetes.io/projected/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-kube-api-access-vpjcn\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.450063 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.450025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:29.540917 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.540885    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:29.632189 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.631365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:29.858393 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:29.858358    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"]
Oct 30 12:57:29.881567 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:29.881532    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ee4d93_f69e_4b21_84c9_c3cfce9dcdfa.slice/crio-8ce437055d0458893d41311eb3c61791c6d93b3f83bf6e99a5ba01816b252668 WatchSource:0}: Error finding container 8ce437055d0458893d41311eb3c61791c6d93b3f83bf6e99a5ba01816b252668: Status 404 returned error can't find the container with id 8ce437055d0458893d41311eb3c61791c6d93b3f83bf6e99a5ba01816b252668
Oct 30 12:57:30.121764 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:30.121681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:30.387172 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:30.387092    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qlp5s" event={"ID":"4a9daaf5-4ea6-472e-9972-68cfeee5ae16","Type":"ContainerStarted","Data":"04606d64551dffe6b5a90489a78bdd807449165f8c8b6ba85a0e38c626be1ddb"}
Oct 30 12:57:30.389016 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:30.388991    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqj4w" event={"ID":"793c2518-45fa-46e0-9a23-766ae40d922b","Type":"ContainerStarted","Data":"4ad2c064633c3ae5f3ec7fd004551890cda4297c9b44e85e6c565532373c3278"}
Oct 30 12:57:30.389193 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:30.389036    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fqj4w" event={"ID":"793c2518-45fa-46e0-9a23-766ae40d922b","Type":"ContainerStarted","Data":"f2ee73c07b41feeb0958b32be5ed4254d97885ce9e518e874ff387c9566d9191"}
Oct 30 12:57:30.389193 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:30.389067    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:30.390203 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:30.390178    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df" event={"ID":"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa","Type":"ContainerStarted","Data":"8ce437055d0458893d41311eb3c61791c6d93b3f83bf6e99a5ba01816b252668"}
Oct 30 12:57:30.434099 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:30.434050    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qlp5s" podStartSLOduration=2.2968880130000002 podStartE2EDuration="4.43403342s" totalImagesPullingTime="2.137145407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:57:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:30.432964754 +0000 UTC m=+61.882041365" watchObservedRunningTime="2026-10-30 12:57:30.43403342 +0000 UTC m=+61.883110031"
Oct 30 12:57:30.477567 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:30.477525    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-fqj4w" podStartSLOduration=2.528144202 podStartE2EDuration="4.477511694s" totalImagesPullingTime="1.949367492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:30.476595874 +0000 UTC m=+61.925672485" watchObservedRunningTime="2026-10-30 12:57:30.477511694 +0000 UTC m=+61.926588303"
Oct 30 12:57:31.394049 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:31.393974    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gphtb" event={"ID":"b5c3d5df-2428-490f-925a-f67ef2e8c0b6","Type":"ContainerStarted","Data":"35c3be73a44b9c4af06f7fe09f9f28f447a0c773e7442ab9cb9d0f42a4006ad6"}
Oct 30 12:57:31.395773 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:31.395739    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df" event={"ID":"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa","Type":"ContainerStarted","Data":"c45fe2ca900e4bf60404a64a740336f2a998747569d838ebf5ebcf199fc270ee"}
Oct 30 12:57:31.395874 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:31.395796    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df" event={"ID":"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa","Type":"ContainerStarted","Data":"e9ef4daa3c813faf18d3fc288c6d0e32f0f3ca96383db55e78f4080bc69f2270"}
Oct 30 12:57:31.437725 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:31.437685    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gphtb" podStartSLOduration=2.592700584 podStartE2EDuration="5.437671874s" totalImagesPullingTime="2.84497129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:31.436734476 +0000 UTC m=+62.885811086" watchObservedRunningTime="2026-10-30 12:57:31.437671874 +0000 UTC m=+62.886748481"
Oct 30 12:57:31.481625 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:31.481577    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df" podStartSLOduration=1.372250593 podStartE2EDuration="2.481563652s" totalImagesPullingTime="1.109313059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:31.480393454 +0000 UTC m=+62.929470063" watchObservedRunningTime="2026-10-30 12:57:31.481563652 +0000 UTC m=+62.930640262"
Oct 30 12:57:31.840002 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:31.839969    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:33.592254 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.592222    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:33.807033 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.806999    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:33.865117 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.865041    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"]
Oct 30 12:57:33.868261 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.868245    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:33.886826 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.886803    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 12:57:33.887633 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.887616    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 12:57:33.902706 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.902673    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"]
Oct 30 12:57:33.905812 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.905792    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bkk45"]
Oct 30 12:57:33.906050 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.906027    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:33.908765 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.908746    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"]
Oct 30 12:57:33.908868 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.908831    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:33.924733 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.924712    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"]
Oct 30 12:57:33.950524 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.950495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6214326d-31ce-4577-bd4b-7997c910a058-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:33.950676 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.950533    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:33.950676 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.950663    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:33.950781 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.950689    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4bf5\" (UniqueName: \"kubernetes.io/projected/6214326d-31ce-4577-bd4b-7997c910a058-kube-api-access-w4bf5\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:33.969520 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.969492    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:33.978533 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.978511    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 12:57:33.978533 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.978512    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 12:57:33.978674 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.978545    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-djsc9\""
Oct 30 12:57:33.978674 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.978584    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 12:57:33.978674 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.978646    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-skpfr\""
Oct 30 12:57:33.978786 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.978713    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 12:57:33.978975 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:33.978957    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-h2dkw\""
Oct 30 12:57:34.009370 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.009350    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 12:57:34.009519 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.009496    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 12:57:34.051566 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051543    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.051689 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051572    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-metrics-client-ca\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.051689 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051597    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:34.051689 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4bf5\" (UniqueName: \"kubernetes.io/projected/6214326d-31ce-4577-bd4b-7997c910a058-kube-api-access-w4bf5\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:34.051689 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051670    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.051895 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051719    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6214326d-31ce-4577-bd4b-7997c910a058-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:34.051895 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051757    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:34.051895 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051784    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.051895 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051817    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br287\" (UniqueName: \"kubernetes.io/projected/1a451455-21be-4f3a-a56d-8c4026bcf654-kube-api-access-br287\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.051895 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051846    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-tls\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.052154 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051942    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.052154 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.051999    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-sys\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.052154 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.052031    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.052154 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.052081    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-wtmp\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.052360 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.052151    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-textfile\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.052360 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.052271    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-root\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.052360 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.052345    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.052500 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.052415    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb64l\" (UniqueName: \"kubernetes.io/projected/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-api-access-jb64l\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.052500 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.052458    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6214326d-31ce-4577-bd4b-7997c910a058-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:34.052500 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.052461    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.054026 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.054007    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:34.054119 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.054102    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:34.076912 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.076886    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4bf5\" (UniqueName: \"kubernetes.io/projected/6214326d-31ce-4577-bd4b-7997c910a058-kube-api-access-w4bf5\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:34.129735 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.129668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:34.152762 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.152735    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.152848 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.152780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.152848 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.152806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br287\" (UniqueName: \"kubernetes.io/projected/1a451455-21be-4f3a-a56d-8c4026bcf654-kube-api-access-br287\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.152848 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.152833    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-tls\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153005 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.152862    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.153005 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.152888    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-sys\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153005 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.152914    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153005 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.152988    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-sys\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153194 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153071    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-wtmp\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153194 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153112    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-textfile\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153194 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-root\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153194 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153424 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153213    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb64l\" (UniqueName: \"kubernetes.io/projected/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-api-access-jb64l\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.153424 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.153519 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153450    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-wtmp\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153519 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153488    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-textfile\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153620 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153542    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-root\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.153620 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153576    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.153620 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153576    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.153804 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153304    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.153870 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153851    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.153964 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153941    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.154022 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.153992    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-metrics-client-ca\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.154254 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.154238    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-metrics-client-ca\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.155375 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.155350    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.155714 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.155695    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-tls\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.155768 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.155736    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.155768 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.155752    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.180445 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.180426    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb64l\" (UniqueName: \"kubernetes.io/projected/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-api-access-jb64l\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.180507 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.180462    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-br287\" (UniqueName: \"kubernetes.io/projected/1a451455-21be-4f3a-a56d-8c4026bcf654-kube-api-access-br287\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.183358 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.183337    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:34.221396 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.221367    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:34.230013 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.229759    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:34.243707 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:34.243670    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a451455_21be_4f3a_a56d_8c4026bcf654.slice/crio-a352edd24d84cc0fc288b36c7c48ce15dc2cd019d4e5bc396c8b242a82bc43c8 WatchSource:0}: Error finding container a352edd24d84cc0fc288b36c7c48ce15dc2cd019d4e5bc396c8b242a82bc43c8: Status 404 returned error can't find the container with id a352edd24d84cc0fc288b36c7c48ce15dc2cd019d4e5bc396c8b242a82bc43c8
Oct 30 12:57:34.330887 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.330858    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:34.406001 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.405920    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkk45" event={"ID":"1a451455-21be-4f3a-a56d-8c4026bcf654","Type":"ContainerStarted","Data":"a352edd24d84cc0fc288b36c7c48ce15dc2cd019d4e5bc396c8b242a82bc43c8"}
Oct 30 12:57:34.499108 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.499077    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"]
Oct 30 12:57:34.501692 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:34.501658    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6214326d_31ce_4577_bd4b_7997c910a058.slice/crio-c86ff0970a9bb85977512368549d35042c2899c8ad0c393ea8f2b77e40037c25 WatchSource:0}: Error finding container c86ff0970a9bb85977512368549d35042c2899c8ad0c393ea8f2b77e40037c25: Status 404 returned error can't find the container with id c86ff0970a9bb85977512368549d35042c2899c8ad0c393ea8f2b77e40037c25
Oct 30 12:57:34.540587 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.540565    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"]
Oct 30 12:57:34.543972 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:34.543947    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc420e84e_1a2d_4baa_84cd_b6cabe7ec4d4.slice/crio-d38c492d5386de0ea7cf31146fd539f68853b5d21dd5cd56353205012ca570f4 WatchSource:0}: Error finding container d38c492d5386de0ea7cf31146fd539f68853b5d21dd5cd56353205012ca570f4: Status 404 returned error can't find the container with id d38c492d5386de0ea7cf31146fd539f68853b5d21dd5cd56353205012ca570f4
Oct 30 12:57:34.636661 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.636634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:34.801137 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.801100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:34.860672 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.860635    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:34.860828 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.860689    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:34.876230 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.876203    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 12:57:34.876386 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.876203    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 12:57:34.883997 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.883970    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:34.884118 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.884068    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:34.952278 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.952243    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:34.961785 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.961401    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:34.976696 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.976669    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 12:57:34.997020 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:34.996991    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 12:57:35.005199 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.005144    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:35.030126 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.030097    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v4tv9\""
Oct 30 12:57:35.035199 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.035174    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:35.043647 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.043625    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-sqtw6\""
Oct 30 12:57:35.048463 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.048443    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:35.063399 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.063379    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vg7bt\""
Oct 30 12:57:35.069605 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.068746    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:35.108966 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.105106    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:35.258378 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.258347    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 12:57:35.261823 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.261800    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.266125 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.266072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:35.279178 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279156    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 12:57:35.279272 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279214    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 12:57:35.279541 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279517    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 12:57:35.279598 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279575    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 12:57:35.279694 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279678    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 12:57:35.279783 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279770    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-b82bm\""
Oct 30 12:57:35.279868 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279855    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 12:57:35.280005 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279888    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 12:57:35.280005 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279894    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 12:57:35.280005 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.279955    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 12:57:35.308144 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.307918    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 12:57:35.364710 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.364664    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.364863 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.364721    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.364863 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.364841    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.364985 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.364886    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.364985 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.364916    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.364985 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.364947    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.365129 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.364997    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.365129 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.365021    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.365129 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.365043    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.365129 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.365079    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.365129 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.365113    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nhbx\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.365381 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.365138    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.365381 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.365165    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.374229 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.374201    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xtz6h"]
Oct 30 12:57:35.377799 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:35.377768    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1062c6b4_6828_4e92_92a5_c78eccb1fe26.slice/crio-aeb7cb636cec5f7748fdc9f1f81dff156ae49a67936db2ad35181d08645b8831 WatchSource:0}: Error finding container aeb7cb636cec5f7748fdc9f1f81dff156ae49a67936db2ad35181d08645b8831: Status 404 returned error can't find the container with id aeb7cb636cec5f7748fdc9f1f81dff156ae49a67936db2ad35181d08645b8831
Oct 30 12:57:35.399815 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.399638    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gkrqb"]
Oct 30 12:57:35.402249 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:35.402224    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69d9666_d791_48fe_aad3_7a91f2ab2d2c.slice/crio-88dac2fc3b5fcc3a3c2de4bb891256e0babeeaa3046db943f0c7ebb0358a3228 WatchSource:0}: Error finding container 88dac2fc3b5fcc3a3c2de4bb891256e0babeeaa3046db943f0c7ebb0358a3228: Status 404 returned error can't find the container with id 88dac2fc3b5fcc3a3c2de4bb891256e0babeeaa3046db943f0c7ebb0358a3228
Oct 30 12:57:35.411116 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.411077    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh" event={"ID":"6214326d-31ce-4577-bd4b-7997c910a058","Type":"ContainerStarted","Data":"dcb727a20eecd18e05e96b68dba9453bab9c95ada6e285f413719f6df15d53f1"}
Oct 30 12:57:35.411216 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.411155    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh" event={"ID":"6214326d-31ce-4577-bd4b-7997c910a058","Type":"ContainerStarted","Data":"9b2e84a5cb4cd51793e6807321795ab1e4b6e416bd2f2e29722f66a4677f3780"}
Oct 30 12:57:35.411216 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.411188    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh" event={"ID":"6214326d-31ce-4577-bd4b-7997c910a058","Type":"ContainerStarted","Data":"c86ff0970a9bb85977512368549d35042c2899c8ad0c393ea8f2b77e40037c25"}
Oct 30 12:57:35.412747 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.412725    2583 generic.go:356] "Generic (PLEG): container finished" podID="1a451455-21be-4f3a-a56d-8c4026bcf654" containerID="524e08660622872a417606f8c83934b6effa70e48378912d8b2c04b49c516578" exitCode=0
Oct 30 12:57:35.412866 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.412811    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkk45" event={"ID":"1a451455-21be-4f3a-a56d-8c4026bcf654","Type":"ContainerDied","Data":"524e08660622872a417606f8c83934b6effa70e48378912d8b2c04b49c516578"}
Oct 30 12:57:35.414003 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.413977    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xtz6h" event={"ID":"1062c6b4-6828-4e92-92a5-c78eccb1fe26","Type":"ContainerStarted","Data":"aeb7cb636cec5f7748fdc9f1f81dff156ae49a67936db2ad35181d08645b8831"}
Oct 30 12:57:35.415386 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.415364    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2" event={"ID":"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4","Type":"ContainerStarted","Data":"d38c492d5386de0ea7cf31146fd539f68853b5d21dd5cd56353205012ca570f4"}
Oct 30 12:57:35.416614 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.416591    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gkrqb" event={"ID":"c69d9666-d791-48fe-aad3-7a91f2ab2d2c","Type":"ContainerStarted","Data":"88dac2fc3b5fcc3a3c2de4bb891256e0babeeaa3046db943f0c7ebb0358a3228"}
Oct 30 12:57:35.433183 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.432158    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xc6hk"]
Oct 30 12:57:35.435793 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.435767    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:35.443079 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:35.443042    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4faa20e7_dd05_4416_9b9e_ed52b91d388d.slice/crio-62bb2d3a02381a4358ccbf6e514547ed66f9256697136170e2ee1af5d74ab0a7 WatchSource:0}: Error finding container 62bb2d3a02381a4358ccbf6e514547ed66f9256697136170e2ee1af5d74ab0a7: Status 404 returned error can't find the container with id 62bb2d3a02381a4358ccbf6e514547ed66f9256697136170e2ee1af5d74ab0a7
Oct 30 12:57:35.466505 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466473    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466663 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466522    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466663 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466554    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466663 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466592    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466663 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466663 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466917 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466917 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466697    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nhbx\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466917 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466719    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466917 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466744    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466917 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466769    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466917 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466809    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.466917 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.466844    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.467875 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.467821    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.469222 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.469198    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.470651 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.470623    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.471124 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.471100    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.472840 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.472785    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.474036 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.473982    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.474668 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.474615    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.475229 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.475204    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.475229 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.475211    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.475395 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.475363    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.475449 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.475413    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.477348 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.477330    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.493630 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.493580    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nhbx\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.593369 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.593341    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:35.950040 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:35.950015    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 12:57:35.967501 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:35.967474    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0855ce73_dcfd_4398_b658_301a73ea990d.slice/crio-cd49119309dcb11aab8d8f869b8f8dc0fae222844bc784f4d5ae31d20b599ffb WatchSource:0}: Error finding container cd49119309dcb11aab8d8f869b8f8dc0fae222844bc784f4d5ae31d20b599ffb: Status 404 returned error can't find the container with id cd49119309dcb11aab8d8f869b8f8dc0fae222844bc784f4d5ae31d20b599ffb
Oct 30 12:57:36.094597 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.094570    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"]
Oct 30 12:57:36.098804 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.098783    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.121063 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.120591    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 12:57:36.121063 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.120726    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2665n\""
Oct 30 12:57:36.121063 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.120836    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 12:57:36.121063 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.120902    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 12:57:36.121063 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.120955    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 12:57:36.121063 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.120993    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 12:57:36.121063 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.121042    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-59dum1jo40hp4\""
Oct 30 12:57:36.133149 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.133072    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"]
Oct 30 12:57:36.172936 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.172624    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.172936 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.172661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-grpc-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.172936 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.172683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hskxz\" (UniqueName: \"kubernetes.io/projected/99f8c0d1-d514-488c-9401-b745e93ad9ac-kube-api-access-hskxz\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.172936 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.172703    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.172936 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.172724    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.172936 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.172744    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.172936 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.172760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99f8c0d1-d514-488c-9401-b745e93ad9ac-tmp\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.172936 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.172785    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f8c0d1-d514-488c-9401-b745e93ad9ac-metrics-client-ca\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.172936 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.172805    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.274160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.273584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.274160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.273646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.274160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.273673    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99f8c0d1-d514-488c-9401-b745e93ad9ac-tmp\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.274160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.273718    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f8c0d1-d514-488c-9401-b745e93ad9ac-metrics-client-ca\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.274160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.273752    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.274160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.273813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.274160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.273847    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-grpc-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.274160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.273899    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hskxz\" (UniqueName: \"kubernetes.io/projected/99f8c0d1-d514-488c-9401-b745e93ad9ac-kube-api-access-hskxz\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.274160 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.273938    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.275456 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.275164    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99f8c0d1-d514-488c-9401-b745e93ad9ac-tmp\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.276014 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.275992    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f8c0d1-d514-488c-9401-b745e93ad9ac-metrics-client-ca\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.281192 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.281165    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.281331 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.281203    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.281632 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.281592    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.282980 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.282960    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.284265 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.284242    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.285898 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.285874    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-grpc-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.305142 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.305113    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hskxz\" (UniqueName: \"kubernetes.io/projected/99f8c0d1-d514-488c-9401-b745e93ad9ac-kube-api-access-hskxz\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.425424 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.425380    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2" event={"ID":"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4","Type":"ContainerStarted","Data":"36b657b1cd9de602a47c9c236ba29800889a877063e9acc6780cb8a8072d98da"}
Oct 30 12:57:36.425604 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.425447    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2" event={"ID":"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4","Type":"ContainerStarted","Data":"ca70d4daffe008c80935eb7a2a9a210f6a9b21730c65113355e1a3b79b9aac30"}
Oct 30 12:57:36.425604 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.425477    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2" event={"ID":"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4","Type":"ContainerStarted","Data":"12d1e523d49dffff5259e3790f774bd1604e15dfaa9dded630ecb145d640b727"}
Oct 30 12:57:36.429206 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.429180    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh" event={"ID":"6214326d-31ce-4577-bd4b-7997c910a058","Type":"ContainerStarted","Data":"3cbb00a30fd0c4dee959f9cae193bc3567bc1a34309ce51667503309a8afdf40"}
Oct 30 12:57:36.434604 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.434524    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkk45" event={"ID":"1a451455-21be-4f3a-a56d-8c4026bcf654","Type":"ContainerStarted","Data":"5da666d76e43f9f4904c263f5d3781cc696d2ef28635d0fe421e0fd7e7aebdf0"}
Oct 30 12:57:36.434604 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.434574    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bkk45" event={"ID":"1a451455-21be-4f3a-a56d-8c4026bcf654","Type":"ContainerStarted","Data":"3a8c7ad355071b620309ca6febbda68ecd9438f27799c2551ab732b62bdd2ca2"}
Oct 30 12:57:36.436320 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.436117    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:36.436864 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.436779    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerStarted","Data":"cd49119309dcb11aab8d8f869b8f8dc0fae222844bc784f4d5ae31d20b599ffb"}
Oct 30 12:57:36.438090 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.438041    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xc6hk" event={"ID":"4faa20e7-dd05-4416-9b9e-ed52b91d388d","Type":"ContainerStarted","Data":"62bb2d3a02381a4358ccbf6e514547ed66f9256697136170e2ee1af5d74ab0a7"}
Oct 30 12:57:36.477257 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.476910    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2" podStartSLOduration=2.100435564 podStartE2EDuration="3.476890546s" totalImagesPullingTime="1.376454982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:36.47190483 +0000 UTC m=+67.920981451" watchObservedRunningTime="2026-10-30 12:57:36.476890546 +0000 UTC m=+67.925967157"
Oct 30 12:57:36.568959 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.568898    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh" podStartSLOduration=2.430386811 podStartE2EDuration="3.568880683s" totalImagesPullingTime="1.138493872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:36.521970788 +0000 UTC m=+67.971047397" watchObservedRunningTime="2026-10-30 12:57:36.568880683 +0000 UTC m=+68.017957295"
Oct 30 12:57:36.570082 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.570039    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bkk45" podStartSLOduration=2.667539009 podStartE2EDuration="3.570024846s" totalImagesPullingTime="822.598684ms" totalInitContainerRuntime="79.887153ms" isStatefulPod=true podCreationTimestamp="2026-10-30 12:57:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:36.569591394 +0000 UTC m=+68.018668006" watchObservedRunningTime="2026-10-30 12:57:36.570024846 +0000 UTC m=+68.019101458"
Oct 30 12:57:36.773371 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.773253    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"]
Oct 30 12:57:36.777465 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:36.776110    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99f8c0d1_d514_488c_9401_b745e93ad9ac.slice/crio-9a9f8ba48bd42ba5051e5e2d12b62c1b47447d2b9c2eb5eca82a2e2999e4a534 WatchSource:0}: Error finding container 9a9f8ba48bd42ba5051e5e2d12b62c1b47447d2b9c2eb5eca82a2e2999e4a534: Status 404 returned error can't find the container with id 9a9f8ba48bd42ba5051e5e2d12b62c1b47447d2b9c2eb5eca82a2e2999e4a534
Oct 30 12:57:36.885839 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:36.885777    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:37.050730 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:37.050656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:37.241588 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:37.241564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:37.398368 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:37.398283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:37.444305 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:37.444246    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xtz6h" event={"ID":"1062c6b4-6828-4e92-92a5-c78eccb1fe26","Type":"ContainerStarted","Data":"7e838ae6df48a0e703a6d07ffd895d5f8d70c6cab6f09bcc0ced0d86baefc76e"}
Oct 30 12:57:37.444489 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:37.444325    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xtz6h" event={"ID":"1062c6b4-6828-4e92-92a5-c78eccb1fe26","Type":"ContainerStarted","Data":"b043fa118e7b9cf3e3d4e7579e059da3a37fc03f747876789c3557869965143a"}
Oct 30 12:57:37.446519 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:37.446430    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c" event={"ID":"99f8c0d1-d514-488c-9401-b745e93ad9ac","Type":"ContainerStarted","Data":"9a9f8ba48bd42ba5051e5e2d12b62c1b47447d2b9c2eb5eca82a2e2999e4a534"}
Oct 30 12:57:37.448784 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:37.448755    2583 generic.go:356] "Generic (PLEG): container finished" podID="0855ce73-dcfd-4398-b658-301a73ea990d" containerID="7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f" exitCode=0
Oct 30 12:57:37.450914 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:37.449878    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerDied","Data":"7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f"}
Oct 30 12:57:37.488270 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:37.488216    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xtz6h" podStartSLOduration=67.21379913 podStartE2EDuration="1m8.488201205s" totalImagesPullingTime="1.274402075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:37.488097621 +0000 UTC m=+68.937174232" watchObservedRunningTime="2026-10-30 12:57:37.488201205 +0000 UTC m=+68.937277814"
Oct 30 12:57:38.452674 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.451324    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"]
Oct 30 12:57:38.456046 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.455061    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.485877 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.485844    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 12:57:38.485877 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.485847    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 12:57:38.486070 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.485855    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 12:57:38.486070 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.485873    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5ikgvvfbsde4u\""
Oct 30 12:57:38.486070 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.485921    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pd6vw\""
Oct 30 12:57:38.486070 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.485998    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 12:57:38.493081 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.493057    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"]
Oct 30 12:57:38.594956 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.594924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-metrics-server-audit-profiles\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.595137 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.594996    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-tls\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.595137 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.595025    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/88f01f9c-02af-4e17-9a80-7f769221c703-audit-log\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.595137 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.595041    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8jdj\" (UniqueName: \"kubernetes.io/projected/88f01f9c-02af-4e17-9a80-7f769221c703-kube-api-access-t8jdj\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.595137 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.595066    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-client-certs\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.595137 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.595086    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.595137 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.595123    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-client-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.696148 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.696116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.696305 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.696159    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-client-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.696305 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.696202    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-metrics-server-audit-profiles\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.696305 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.696230    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-tls\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.696305 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.696251    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/88f01f9c-02af-4e17-9a80-7f769221c703-audit-log\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.696305 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.696271    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8jdj\" (UniqueName: \"kubernetes.io/projected/88f01f9c-02af-4e17-9a80-7f769221c703-kube-api-access-t8jdj\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.696480 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.696309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-client-certs\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.697810 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.697168    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.697810 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.697456    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/88f01f9c-02af-4e17-9a80-7f769221c703-audit-log\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.698463 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.698437    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-metrics-server-audit-profiles\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.699619 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.699571    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-client-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.700349 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.700275    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-client-certs\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.701471 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.701450    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-tls\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.703810 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.703762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:38.706900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.706883    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"]
Oct 30 12:57:38.711427 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.711410    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:38.737913 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.737888    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"]
Oct 30 12:57:38.738023 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.737946    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8jdj\" (UniqueName: \"kubernetes.io/projected/88f01f9c-02af-4e17-9a80-7f769221c703-kube-api-access-t8jdj\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.762816 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.762792    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 12:57:38.763038 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.763021    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6gqfl\""
Oct 30 12:57:38.770563 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.770539    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:38.800435 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.800406    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpbk9\" (UniqueName: \"kubernetes.io/projected/b8cc6455-76b2-40a2-aa43-5b516d85d543-kube-api-access-wpbk9\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:38.800549 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.800480    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8cc6455-76b2-40a2-aa43-5b516d85d543-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:38.901494 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.901427    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpbk9\" (UniqueName: \"kubernetes.io/projected/b8cc6455-76b2-40a2-aa43-5b516d85d543-kube-api-access-wpbk9\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:38.901948 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.901925    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8cc6455-76b2-40a2-aa43-5b516d85d543-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:38.905248 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.905201    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8cc6455-76b2-40a2-aa43-5b516d85d543-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:38.909597 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.909574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:38.927648 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:38.927585    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpbk9\" (UniqueName: \"kubernetes.io/projected/b8cc6455-76b2-40a2-aa43-5b516d85d543-kube-api-access-wpbk9\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:39.030231 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:39.030156    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:39.114307 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:39.114255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:39.117673 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:39.117647    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"]
Oct 30 12:57:39.130433 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:39.130403    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f01f9c_02af_4e17_9a80_7f769221c703.slice/crio-26b4d4e3bf0dc818e862ac74b01999e2ea17b713606006528313d9f86714c102 WatchSource:0}: Error finding container 26b4d4e3bf0dc818e862ac74b01999e2ea17b713606006528313d9f86714c102: Status 404 returned error can't find the container with id 26b4d4e3bf0dc818e862ac74b01999e2ea17b713606006528313d9f86714c102
Oct 30 12:57:39.348693 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:39.348578    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"]
Oct 30 12:57:39.352015 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:39.351983    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8cc6455_76b2_40a2_aa43_5b516d85d543.slice/crio-7050d3799203f75c4bc749324f41dfa9d14fb188f53510c4ec1af62dbdc7287a WatchSource:0}: Error finding container 7050d3799203f75c4bc749324f41dfa9d14fb188f53510c4ec1af62dbdc7287a: Status 404 returned error can't find the container with id 7050d3799203f75c4bc749324f41dfa9d14fb188f53510c4ec1af62dbdc7287a
Oct 30 12:57:39.461900 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:39.461865    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k" event={"ID":"88f01f9c-02af-4e17-9a80-7f769221c703","Type":"ContainerStarted","Data":"26b4d4e3bf0dc818e862ac74b01999e2ea17b713606006528313d9f86714c102"}
Oct 30 12:57:39.463556 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:39.463525    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5" event={"ID":"b8cc6455-76b2-40a2-aa43-5b516d85d543","Type":"ContainerStarted","Data":"7050d3799203f75c4bc749324f41dfa9d14fb188f53510c4ec1af62dbdc7287a"}
Oct 30 12:57:40.404644 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.403720    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:40.481578 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.481374    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerStarted","Data":"8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b"}
Oct 30 12:57:40.481578 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.481453    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerStarted","Data":"b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069"}
Oct 30 12:57:40.481578 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.481487    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerStarted","Data":"4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3"}
Oct 30 12:57:40.481578 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.481512    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerStarted","Data":"308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a"}
Oct 30 12:57:40.481578 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.481536    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerStarted","Data":"cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb"}
Oct 30 12:57:40.483557 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.483529    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xc6hk" event={"ID":"4faa20e7-dd05-4416-9b9e-ed52b91d388d","Type":"ContainerStarted","Data":"fdc055ac0e3ab2b756bcd542c12a6c9a99df02f09c16bb53be85b230d5abf3db"}
Oct 30 12:57:40.484339 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.484309    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:40.487164 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.486951    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c" event={"ID":"99f8c0d1-d514-488c-9401-b745e93ad9ac","Type":"ContainerStarted","Data":"e76f84f255038ba6f0ea5c55461f393e15afd1fc4d6662cc2e1c377887604876"}
Oct 30 12:57:40.487164 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.487020    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c" event={"ID":"99f8c0d1-d514-488c-9401-b745e93ad9ac","Type":"ContainerStarted","Data":"0b22658f6bdb10219d3387f028d54141a2446327ab1af60ef911035c478f2e8b"}
Oct 30 12:57:40.487164 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.487090    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c" event={"ID":"99f8c0d1-d514-488c-9401-b745e93ad9ac","Type":"ContainerStarted","Data":"a0ea267a08e8b5d9d7184297c7c8a3d9a6d8da11431735a9953dcffe26de3aec"}
Oct 30 12:57:40.542374 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.542274    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xc6hk" podStartSLOduration=67.089648546 podStartE2EDuration="1m11.542254427s" totalImagesPullingTime="4.452605881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:40.541446449 +0000 UTC m=+71.990523060" watchObservedRunningTime="2026-10-30 12:57:40.542254427 +0000 UTC m=+71.991331040"
Oct 30 12:57:40.895753 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.895728    2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 12:57:40.901980 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.901959    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:40.939449 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.939028    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 12:57:40.939449 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.939199    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 12:57:40.939771 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.939750    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 12:57:40.956394 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.956365    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:40.973643 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.973619    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 12:57:40.974147 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.974128    2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.975532    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.975583    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.975694    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.976017    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-82464\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.976115    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.976149    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.976256    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.976311    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.976418    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cacau0ihglokf\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.976464    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.976516    2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 12:57:40.976547 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:40.976528    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 12:57:41.027447 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.027270    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.027447 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.027333    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.027447 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.027369    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028181 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.027392    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028257 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028237    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028342 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028272    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgd46\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028342 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028334    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028432 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028368    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028432 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028398    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028432 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028429    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028524 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028443    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028524 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028462    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028524 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028493    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028524 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028511    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028524 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028524    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028662 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028553    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028662 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028590    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028662 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028748 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028688    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.028748 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.028726    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129408 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129375    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129573 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129415    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129573 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129446    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129573 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129484    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129573 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129506    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129573 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129527    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129573 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129847 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129847 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129617    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129847 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129847 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129847 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129847 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129847 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129847 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129790    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.129847 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129829    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.130234 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129852    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgd46\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.130234 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.130234 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129916    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.130234 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.129941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.131238 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.130598    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.131625 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.131593    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.133255 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.132079    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.133255 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.132958    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.133577 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.133551    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.134587 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.134218    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.136013 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.135787    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.136517 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.136470    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.137664 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.137637    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.137749 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.137724    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.138899 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.138860    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.141322 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.140886    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.141322 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.140901    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.141322 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.141186    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.141322 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.141282    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.141639 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.141601    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.142481 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.142457    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.142633 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.142616    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.144070 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.144030    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.159728 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.159686    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgd46\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.271052 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.271023    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:41.473812 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.473779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:41.494206 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.494166    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5" event={"ID":"b8cc6455-76b2-40a2-aa43-5b516d85d543","Type":"ContainerStarted","Data":"f800df7be0132f7b2f5f0b02efbd9e401cb8b41f37399a1bf88c27ad374cd8d6"}
Oct 30 12:57:41.494701 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.494535    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:41.499744 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.499717    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerStarted","Data":"ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240"}
Oct 30 12:57:41.500419 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.500395    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:41.502251 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.501810    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k" event={"ID":"88f01f9c-02af-4e17-9a80-7f769221c703","Type":"ContainerStarted","Data":"35d0af613e15a4447fed2366e8cd35cdaa929cbf4f35c0afb76c438983c9f1d1"}
Oct 30 12:57:41.506418 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.506263    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c" event={"ID":"99f8c0d1-d514-488c-9401-b745e93ad9ac","Type":"ContainerStarted","Data":"ddde79603c3041f5af9e7ceeac01ce7c7739df0301799adaa45d362b25268233"}
Oct 30 12:57:41.506418 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.506340    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c" event={"ID":"99f8c0d1-d514-488c-9401-b745e93ad9ac","Type":"ContainerStarted","Data":"5d9762c085b9aebae7cdb5335ac6375b543ec8117d172385303c224fd47d2a6f"}
Oct 30 12:57:41.506418 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.506368    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c" event={"ID":"99f8c0d1-d514-488c-9401-b745e93ad9ac","Type":"ContainerStarted","Data":"32e5e8ee9320184d012034e28e2413828d4d51d455d456752d5964ec4b00bc82"}
Oct 30 12:57:41.506620 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.506492    2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:41.598811 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.598703    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5" podStartSLOduration=1.6633577050000001 podStartE2EDuration="3.598686072s" totalImagesPullingTime="1.935328367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:41.538989667 +0000 UTC m=+72.988066277" watchObservedRunningTime="2026-10-30 12:57:41.598686072 +0000 UTC m=+73.047762686"
Oct 30 12:57:41.600113 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.600058    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.412621261 podStartE2EDuration="6.600046696s" totalImagesPullingTime="4.173921223s" totalInitContainerRuntime="13.504212ms" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:41.598214977 +0000 UTC m=+73.047291589" watchObservedRunningTime="2026-10-30 12:57:41.600046696 +0000 UTC m=+73.049123311"
Oct 30 12:57:41.626945 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.626917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:41.650069 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.650043    2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 12:57:41.653675 ip-10-0-130-180 kubenswrapper[2583]: W1030 12:57:41.653546    2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod221f1eb2_45dc_4f3e_b7d3_8487549100b4.slice/crio-e93a9a65d105becd00d0094f7a53873f2c021457989e5d1b00755c57d644e8e6 WatchSource:0}: Error finding container e93a9a65d105becd00d0094f7a53873f2c021457989e5d1b00755c57d644e8e6: Status 404 returned error can't find the container with id e93a9a65d105becd00d0094f7a53873f2c021457989e5d1b00755c57d644e8e6
Oct 30 12:57:41.660231 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.658411    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c" podStartSLOduration=1.8803806239999998 podStartE2EDuration="5.658380338s" totalImagesPullingTime="3.777999714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:41.656587434 +0000 UTC m=+73.105664067" watchObservedRunningTime="2026-10-30 12:57:41.658380338 +0000 UTC m=+73.107456949"
Oct 30 12:57:41.704445 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:41.704390    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k" podStartSLOduration=1.5965079439999998 podStartE2EDuration="3.704372113s" totalImagesPullingTime="2.107864169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:41.702701888 +0000 UTC m=+73.151778499" watchObservedRunningTime="2026-10-30 12:57:41.704372113 +0000 UTC m=+73.153448723"
Oct 30 12:57:42.512187 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:42.512147    2583 generic.go:356] "Generic (PLEG): container finished" podID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f" exitCode=0
Oct 30 12:57:42.512671 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:42.512300    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerDied","Data":"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"}
Oct 30 12:57:42.512671 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:42.512428    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerStarted","Data":"e93a9a65d105becd00d0094f7a53873f2c021457989e5d1b00755c57d644e8e6"}
Oct 30 12:57:42.514075 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:42.514017    2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gkrqb" event={"ID":"c69d9666-d791-48fe-aad3-7a91f2ab2d2c","Type":"ContainerStarted","Data":"df11c61ab44c2e82b996f09ac7f9eb09ca5fed22e5066f6396478cf8afd24163"}
Oct 30 12:57:42.613385 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:42.612358    2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gkrqb" podStartSLOduration=66.642242892 podStartE2EDuration="1m13.612339004s" totalImagesPullingTime="6.970096112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:42.612212202 +0000 UTC m=+74.061288813" watchObservedRunningTime="2026-10-30 12:57:42.612339004 +0000 UTC m=+74.061415614"
Oct 30 12:57:42.637110 ip-10-0-130-180 kubenswrapper[2583]: I1030 12:57:42.637036    2583 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:57:42.637191 ip-10-0-130-180 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 12:57:42.656793 ip-10-0-130-180 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 12:57:42.656970 ip-10-0-130-180 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 12:57:42.657026 ip-10-0-130-180 systemd[1]: kubelet.service: Consumed 4.385s CPU time.
Oct 30 12:57:42.671126 ip-10-0-130-180 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 12:57:42.731853 ip-10-0-130-180 kubenswrapper[5781]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:57:42.731853 ip-10-0-130-180 kubenswrapper[5781]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 12:57:42.731853 ip-10-0-130-180 kubenswrapper[5781]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:57:42.731853 ip-10-0-130-180 kubenswrapper[5781]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:57:42.734208 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734121    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:42.734208 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734204    5781 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:42.734208 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734210    5781 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:42.734208 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734215    5781 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734219    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734224    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734227    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734232    5781 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734236    5781 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734245    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734249    5781 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734253    5781 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734257    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734262    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734265    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734270    5781 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734276    5781 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734282    5781 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734302    5781 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734307    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734311    5781 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734314    5781 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:42.734511 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734319    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734323    5781 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734326    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734330    5781 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734336    5781 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734343    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734347    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734352    5781 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734356    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734360    5781 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734365    5781 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734370    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734374    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734378    5781 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734383    5781 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734388    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734392    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734403    5781 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734408    5781 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:42.735316 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734412    5781 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734416    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734420    5781 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734424    5781 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734429    5781 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734433    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734437    5781 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734441    5781 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734445    5781 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734449    5781 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734454    5781 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734458    5781 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734462    5781 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734466    5781 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734471    5781 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734475    5781 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734480    5781 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734484    5781 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734488    5781 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734493    5781 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:42.736162 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734497    5781 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734502    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734506    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734510    5781 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734514    5781 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734518    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734522    5781 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734528    5781 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734533    5781 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734538    5781 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734542    5781 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734547    5781 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734553    5781 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734558    5781 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734562    5781 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734566    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734570    5781 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734574    5781 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734579    5781 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734583    5781 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:42.737007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734588    5781 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734592    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734596    5781 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734600    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734605    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734609    5781 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734613    5781 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734618    5781 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734622    5781 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734627    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734631    5781 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734635    5781 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734640    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734644    5781 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734649    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734654    5781 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734658    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734664    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734668    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:42.737886 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734672    5781 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734677    5781 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734681    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734686    5781 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734692    5781 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.734697    5781 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736001    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736011    5781 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736016    5781 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736021    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736028    5781 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736035    5781 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736040    5781 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736045    5781 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736050    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736054    5781 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736059    5781 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736063    5781 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736067    5781 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:42.738721 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736072    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736077    5781 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736081    5781 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736086    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736090    5781 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736095    5781 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736100    5781 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736104    5781 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736108    5781 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736112    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736117    5781 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736123    5781 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736130    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736135    5781 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736140    5781 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736145    5781 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736149    5781 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736153    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736157    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736162    5781 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:42.739617 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736166    5781 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736170    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736174    5781 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736178    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736182    5781 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736187    5781 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736191    5781 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736196    5781 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736200    5781 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736204    5781 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736209    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736214    5781 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736218    5781 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736222    5781 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736227    5781 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736231    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736236    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736240    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736267    5781 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736273    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:42.740495 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736278    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736302    5781 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736307    5781 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736312    5781 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736316    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736321    5781 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736326    5781 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736331    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736335    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736339    5781 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736344    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736349    5781 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736354    5781 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736358    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736362    5781 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736367    5781 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736371    5781 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736376    5781 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736380    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:42.741054 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736385    5781 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736389    5781 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736393    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736398    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736402    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736406    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736410    5781 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736414    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736418    5781 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736423    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736427    5781 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736432    5781 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736436    5781 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736440    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736444    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736448    5781 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736452    5781 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736456    5781 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736461    5781 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736466    5781 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:42.741588 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736471    5781 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736475    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736479    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736483    5781 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736487    5781 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736491    5781 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736496    5781 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736501    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736505    5781 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736509    5781 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736513    5781 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736519    5781 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736524    5781 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.736528    5781 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737352    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737362    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737366    5781 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737371    5781 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737375    5781 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:42.742066 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737379    5781 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737383    5781 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737387    5781 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737392    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737396    5781 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737401    5781 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737405    5781 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737409    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737413    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737417    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737421    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737426    5781 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737433    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737437    5781 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737442    5781 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737446    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737450    5781 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737454    5781 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737458    5781 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:42.742607 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737463    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737467    5781 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737471    5781 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737475    5781 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737480    5781 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737484    5781 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737489    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737493    5781 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737497    5781 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737502    5781 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737506    5781 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737510    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737514    5781 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737518    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737523    5781 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737527    5781 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737531    5781 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737535    5781 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737539    5781 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737543    5781 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:42.743141 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737547    5781 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737551    5781 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737556    5781 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737560    5781 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737565    5781 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737569    5781 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737574    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737578    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737582    5781 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737586    5781 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737590    5781 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737594    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737598    5781 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737602    5781 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737606    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737611    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737615    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737622    5781 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737626    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:42.743932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737631    5781 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737635    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737640    5781 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737644    5781 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737649    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737653    5781 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737657    5781 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737664    5781 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737670    5781 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737675    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737679    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737684    5781 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737688    5781 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737692    5781 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737696    5781 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737700    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737705    5781 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737709    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737714    5781 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:42.744444 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737718    5781 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737723    5781 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737728    5781 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737736    5781 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737742    5781 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737747    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737751    5781 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737756    5781 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737761    5781 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737765    5781 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737769    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737775    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737780    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737785    5781 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737789    5781 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737793    5781 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737798    5781 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737802    5781 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737807    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:42.745204 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737813    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737817    5781 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737821    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737825    5781 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.737830    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.737967    5781 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.737984    5781 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.737993    5781 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738000    5781 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738006    5781 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738012    5781 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738019    5781 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738025    5781 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738031    5781 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738036    5781 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738042    5781 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738048    5781 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738053    5781 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738058    5781 flags.go:64] FLAG: --cgroup-root=""
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738063    5781 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738068    5781 flags.go:64] FLAG: --client-ca-file=""
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738072    5781 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738077    5781 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 12:57:42.746007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738083    5781 flags.go:64] FLAG: --cluster-domain=""
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738090    5781 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738096    5781 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738101    5781 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738106    5781 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738113    5781 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738117    5781 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738123    5781 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738128    5781 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738133    5781 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738137    5781 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738142    5781 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738147    5781 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738152    5781 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738159    5781 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738164    5781 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738169    5781 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738174    5781 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738179    5781 flags.go:64] FLAG: --enable-server="true"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738184    5781 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738191    5781 flags.go:64] FLAG: --event-burst="100"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738196    5781 flags.go:64] FLAG: --event-qps="50"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738201    5781 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738206    5781 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 12:57:42.747001 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738211    5781 flags.go:64] FLAG: --eviction-hard=""
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738218    5781 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738223    5781 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738228    5781 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738233    5781 flags.go:64] FLAG: --eviction-soft=""
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738237    5781 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738242    5781 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738246    5781 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738251    5781 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738258    5781 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738262    5781 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738267    5781 flags.go:64] FLAG: --feature-gates=""
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738273    5781 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738279    5781 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738298    5781 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738304    5781 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738309    5781 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738315    5781 flags.go:64] FLAG: --help="false"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738319    5781 flags.go:64] FLAG: --hostname-override="ip-10-0-130-180.ec2.internal"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738324    5781 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738329    5781 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738334    5781 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738342    5781 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 12:57:42.748068 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738347    5781 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738352    5781 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738357    5781 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738362    5781 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738367    5781 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738372    5781 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738377    5781 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738383    5781 flags.go:64] FLAG: --kube-reserved=""
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738388    5781 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738398    5781 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738404    5781 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738409    5781 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738414    5781 flags.go:64] FLAG: --lock-file=""
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738419    5781 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738424    5781 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738429    5781 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738444    5781 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738449    5781 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738456    5781 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738461    5781 flags.go:64] FLAG: --logging-format="text"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738466    5781 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738472    5781 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738476    5781 flags.go:64] FLAG: --manifest-url=""
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738481    5781 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738488    5781 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 12:57:42.749076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738493    5781 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738499    5781 flags.go:64] FLAG: --max-pods="110"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738504    5781 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738509    5781 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738513    5781 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738518    5781 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738525    5781 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738530    5781 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738534    5781 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738545    5781 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738550    5781 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738555    5781 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738560    5781 flags.go:64] FLAG: --pod-cidr=""
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738565    5781 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738570    5781 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738575    5781 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738579    5781 flags.go:64] FLAG: --port="10250"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738584    5781 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738589    5781 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04b9594dc22ebcc08"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738595    5781 flags.go:64] FLAG: --qos-reserved=""
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738600    5781 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738605    5781 flags.go:64] FLAG: --register-node="true"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738610    5781 flags.go:64] FLAG: --register-with-taints=""
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738616    5781 flags.go:64] FLAG: --registry-burst="10"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738621    5781 flags.go:64] FLAG: --registry-qps="5"
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738627    5781 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 12:57:42.750130 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738631    5781 flags.go:64] FLAG: --reserved-memory=""
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738638    5781 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738642    5781 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738647    5781 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738652    5781 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738657    5781 flags.go:64] FLAG: --runonce="false"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738662    5781 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738667    5781 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738672    5781 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738677    5781 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738681    5781 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738686    5781 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738694    5781 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738698    5781 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738703    5781 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738707    5781 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738712    5781 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738717    5781 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738721    5781 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738726    5781 flags.go:64] FLAG: --system-cgroups=""
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738730    5781 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738738    5781 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738742    5781 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738747    5781 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738753    5781 flags.go:64] FLAG: --tls-min-version=""
Oct 30 12:57:42.751281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738758    5781 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738763    5781 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738768    5781 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738772    5781 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738776    5781 flags.go:64] FLAG: --v="2"
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738784    5781 flags.go:64] FLAG: --version="false"
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738791    5781 flags.go:64] FLAG: --vmodule=""
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738797    5781 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.738803    5781 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739015    5781 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739022    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739027    5781 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739032    5781 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739036    5781 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739041    5781 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739045    5781 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739049    5781 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739054    5781 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739059    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739066    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739070    5781 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739074    5781 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:42.752467 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739079    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739083    5781 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739087    5781 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739091    5781 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739096    5781 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739100    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739104    5781 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739109    5781 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739113    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739118    5781 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739122    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739126    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739131    5781 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739136    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739140    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739144    5781 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739154    5781 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739158    5781 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739163    5781 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:42.753361 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739167    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739172    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739176    5781 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739180    5781 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739184    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739188    5781 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739193    5781 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739197    5781 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739201    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739206    5781 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739211    5781 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739217    5781 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739221    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739225    5781 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739230    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739234    5781 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739238    5781 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739243    5781 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739247    5781 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:42.754075 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739251    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739256    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739260    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739264    5781 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739268    5781 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739272    5781 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739276    5781 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739281    5781 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739304    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739308    5781 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739316    5781 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739320    5781 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739325    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739330    5781 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739334    5781 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739338    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739342    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739348    5781 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739354    5781 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739358    5781 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:42.754694 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739363    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739367    5781 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739372    5781 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739377    5781 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739381    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739385    5781 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739389    5781 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739393    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739398    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739402    5781 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739408    5781 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739414    5781 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739419    5781 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739424    5781 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739428    5781 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739433    5781 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739437    5781 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739441    5781 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739445    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:42.755214 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739450    5781 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739455    5781 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739459    5781 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739466    5781 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739470    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739475    5781 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739479    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739484    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739488    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739492    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739497    5781 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739501    5781 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739505    5781 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739509    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739514    5781 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739518    5781 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:42.755701 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.739566    5781 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739631    5781 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739637    5781 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739642    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739647    5781 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739651    5781 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739655    5781 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739659    5781 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739664    5781 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739668    5781 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739673    5781 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739677    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739682    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739686    5781 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739690    5781 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739694    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739699    5781 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739703    5781 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739708    5781 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739712    5781 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:42.756088 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739718    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739723    5781 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739727    5781 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739731    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739735    5781 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739740    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739745    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739749    5781 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739754    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739758    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739762    5781 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739767    5781 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739772    5781 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739776    5781 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739781    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739785    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739790    5781 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739794    5781 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739799    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:42.756812 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739803    5781 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739807    5781 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739811    5781 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739815    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739820    5781 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739824    5781 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739829    5781 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739833    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739837    5781 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739841    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739846    5781 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739850    5781 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739855    5781 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739861    5781 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739865    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739870    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739874    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739877    5781 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739881    5781 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739885    5781 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:42.757500 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739889    5781 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739893    5781 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739897    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739902    5781 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739910    5781 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739916    5781 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739920    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739924    5781 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739929    5781 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739933    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739937    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739944    5781 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739950    5781 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739956    5781 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739961    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739965    5781 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739969    5781 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739973    5781 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739978    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:42.758002 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739982    5781 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739987    5781 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739991    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.739995    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740000    5781 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740006    5781 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740016    5781 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740022    5781 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740028    5781 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740033    5781 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740037    5781 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740042    5781 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740046    5781 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740050    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740054    5781 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740059    5781 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740063    5781 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740067    5781 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740073    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:42.758510 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740078    5781 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740085    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740089    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740093    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740097    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740102    5781 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740106    5781 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740110    5781 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740114    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.740119    5781 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.746585    5781 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.746613    5781 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.746637    5781 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.746645    5781 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746734    5781 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746742    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746747    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746753    5781 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746757    5781 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:42.758955 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746762    5781 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746767    5781 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746771    5781 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746776    5781 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746780    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746785    5781 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746790    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746794    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746799    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746803    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746807    5781 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746812    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746817    5781 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746822    5781 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746826    5781 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746830    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746837    5781 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746844    5781 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746849    5781 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:42.759456 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746853    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746857    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746862    5781 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746866    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746870    5781 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746875    5781 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746879    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746883    5781 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746887    5781 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746892    5781 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746896    5781 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746900    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746905    5781 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746909    5781 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746913    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746917    5781 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746922    5781 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746926    5781 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746930    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746935    5781 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:42.760063 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746939    5781 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746944    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746948    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746953    5781 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746957    5781 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746961    5781 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746966    5781 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746971    5781 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746975    5781 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746979    5781 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746984    5781 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746988    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746992    5781 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.746997    5781 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747000    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747005    5781 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747009    5781 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747013    5781 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747017    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:42.760921 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747021    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747026    5781 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747030    5781 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747035    5781 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747039    5781 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747044    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747048    5781 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747052    5781 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747056    5781 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747061    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747065    5781 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747069    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747073    5781 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747077    5781 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747081    5781 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747086    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747091    5781 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747096    5781 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747100    5781 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:42.761433 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747104    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747109    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747114    5781 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747118    5781 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747124    5781 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747131    5781 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747136    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747141    5781 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747145    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747150    5781 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747154    5781 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747159    5781 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747163    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747167    5781 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747172    5781 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747176    5781 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747180    5781 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747185    5781 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747189    5781 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747194    5781 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:42.761932 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747198    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747202    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747207    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747211    5781 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.747252    5781 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747535    5781 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747545    5781 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747549    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747553    5781 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747558    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747563    5781 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747568    5781 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747572    5781 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747577    5781 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747582    5781 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747586    5781 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:42.762559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747590    5781 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747595    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747599    5781 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747604    5781 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747608    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747612    5781 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747617    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747621    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747625    5781 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747630    5781 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747635    5781 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747639    5781 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747644    5781 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747648    5781 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747652    5781 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747657    5781 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747661    5781 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747666    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747670    5781 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747674    5781 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:42.763023 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747678    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747682    5781 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747687    5781 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747691    5781 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747695    5781 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747699    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747703    5781 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747709    5781 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747714    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747718    5781 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747725    5781 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747731    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747736    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747741    5781 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747745    5781 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747749    5781 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747754    5781 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747760    5781 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747766    5781 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:42.763555 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747771    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747775    5781 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747780    5781 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747784    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747788    5781 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747793    5781 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747797    5781 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747801    5781 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747805    5781 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747809    5781 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747813    5781 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747817    5781 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747821    5781 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747826    5781 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747831    5781 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747835    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747840    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747844    5781 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747868    5781 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747876    5781 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:42.764030 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747880    5781 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747886    5781 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747890    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747894    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747898    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747902    5781 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747907    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747911    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747916    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747920    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747924    5781 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747928    5781 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747933    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747937    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747941    5781 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747946    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747950    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747954    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747958    5781 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:42.764559 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747963    5781 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747967    5781 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747972    5781 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747976    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747982    5781 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747987    5781 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747993    5781 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.747997    5781 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748002    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748006    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748010    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748014    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748018    5781 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748023    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748027    5781 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748032    5781 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:42.765006 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748037    5781 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.748075    5781 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748126    5781 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748134    5781 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748139    5781 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748144    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748148    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748153    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748157    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748161    5781 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748166    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748170    5781 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748174    5781 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748179    5781 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748183    5781 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748188    5781 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:57:42.765401 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748192    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748196    5781 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748200    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748205    5781 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748209    5781 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748213    5781 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748219    5781 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748224    5781 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748228    5781 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748233    5781 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748237    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748241    5781 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748245    5781 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748249    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748254    5781 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748258    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748263    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748267    5781 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748272    5781 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748277    5781 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:57:42.765787 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748281    5781 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748303    5781 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748307    5781 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748312    5781 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748316    5781 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748320    5781 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748324    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748328    5781 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748333    5781 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748337    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748341    5781 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748346    5781 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748350    5781 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748355    5781 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748359    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748364    5781 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748368    5781 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748372    5781 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748379    5781 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:57:42.766329 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748386    5781 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748392    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748398    5781 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748404    5781 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748409    5781 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748414    5781 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748419    5781 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748425    5781 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748431    5781 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748436    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748443    5781 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748447    5781 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748451    5781 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748456    5781 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748460    5781 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748464    5781 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748469    5781 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748473    5781 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748477    5781 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:57:42.766790 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748481    5781 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748485    5781 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748489    5781 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748493    5781 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748498    5781 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748503    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748507    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748511    5781 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748516    5781 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748520    5781 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748524    5781 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748528    5781 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748534    5781 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748538    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748542    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748547    5781 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748551    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748555    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748560    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:57:42.767307 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748564    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748568    5781 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748572    5781 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748577    5781 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748581    5781 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748587    5781 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748591    5781 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748595    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748600    5781 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748604    5781 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748608    5781 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748612    5781 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748617    5781 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748621    5781 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:57:42.748625    5781 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.748715    5781 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.750220    5781 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.750301    5781 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.750586    5781 server.go:1014] "Starting client certificate rotation"
Oct 30 12:57:42.767776 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.750718    5781 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 12:57:42.768230 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.750776    5781 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 12:51:28 +0000 UTC" deadline="2028-07-22 04:19:25.308333528 +0000 UTC"
Oct 30 12:57:42.768230 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.750810    5781 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15135h21m42.557525026s"
Oct 30 12:57:42.768230 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.751313    5781 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:57:42.768230 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.752960    5781 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:57:42.768230 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.754441    5781 log.go:25] "Validated CRI v1 runtime API"
Oct 30 12:57:42.768230 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.756299    5781 log.go:25] "Validated CRI v1 image API"
Oct 30 12:57:42.768230 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.757014    5781 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 12:57:42.768230 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.759381    5781 fs.go:135] Filesystem UUIDs: map[73a38cac-c335-411e-808a-b3bbe02cef4f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9c053e75-2ec8-4c07-b296-7bbbd1126cda:/dev/nvme0n1p4]
Oct 30 12:57:42.768482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.759403    5781 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0f5ddbb728fbacf85b52607447cfa8da1ebc445a470d9cb53306a0bf051f5009/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0f5ddbb728fbacf85b52607447cfa8da1ebc445a470d9cb53306a0bf051f5009/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1c4544afd27f657628273ca69886f70932c2fa5a4ba34123ba5e5ce43203e091/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1c4544afd27f657628273ca69886f70932c2fa5a4ba34123ba5e5ce43203e091/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/26b4d4e3bf0dc818e862ac74b01999e2ea17b713606006528313d9f86714c102/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/26b4d4e3bf0dc818e862ac74b01999e2ea17b713606006528313d9f86714c102/userdata/shm major:0 minor:386 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4f0d02a497aea0a09e35155819127d1154dc15d08de78b7cf3be97c45a3ae79c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4f0d02a497aea0a09e35155819127d1154dc15d08de78b7cf3be97c45a3ae79c/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/60fa67f9cce7dfdda00d96d1f2f07818fb787f1d8e8951492ba1b524941495ef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/60fa67f9cce7dfdda00d96d1f2f07818fb787f1d8e8951492ba1b524941495ef/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/62bb2d3a02381a4358ccbf6e514547ed66f9256697136170e2ee1af5d74ab0a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/62bb2d3a02381a4358ccbf6e514547ed66f9256697136170e2ee1af5d74ab0a7/userdata/shm major:0 minor:307 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7050d3799203f75c4bc749324f41dfa9d14fb188f53510c4ec1af62dbdc7287a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7050d3799203f75c4bc749324f41dfa9d14fb188f53510c4ec1af62dbdc7287a/userdata/shm major:0 minor:390 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/71446ba892b17259700ac5db04ae23a27f073e8569b9bada878a8427adf79bc9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/71446ba892b17259700ac5db04ae23a27f073e8569b9bada878a8427adf79bc9/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7b63fe8b9bff669d6d61a0846f7fa93f8ac602b5b3f2f2b049b537fa5b2efbf2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7b63fe8b9bff669d6d61a0846f7fa93f8ac602b5b3f2f2b049b537fa5b2efbf2/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8321f08e19f4910820e6872cfb6f88739d17c098e9875cfa591d843b3995946c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8321f08e19f4910820e6872cfb6f88739d17c098e9875cfa591d843b3995946c/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/88dac2fc3b5fcc3a3c2de4bb891256e0babeeaa3046db943f0c7ebb0358a3228/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/88dac2fc3b5fcc3a3c2de4bb891256e0babeeaa3046db943f0c7ebb0358a3228/userdata/shm major:0 minor:304 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8ce437055d0458893d41311eb3c61791c6d93b3f83bf6e99a5ba01816b252668/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8ce437055d0458893d41311eb3c61791c6d93b3f83bf6e99a5ba01816b252668/userdata/shm major:0 minor:233 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/914daae99f98a21338ff4a5f32aba5ef1c8376b8411f53722835adbe3af5a601/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/914daae99f98a21338ff4a5f32aba5ef1c8376b8411f53722835adbe3af5a601/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9a33816ef2cd30b219a540c24f03ef7342a59fb9412e1901e2c6f606ab997294/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9a33816ef2cd30b219a540c24f03ef7342a59fb9412e1901e2c6f606ab997294/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9a9f8ba48bd42ba5051e5e2d12b62c1b47447d2b9c2eb5eca82a2e2999e4a534/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9a9f8ba48bd42ba5051e5e2d12b62c1b47447d2b9c2eb5eca82a2e2999e4a534/userdata/shm major:0 minor:367 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a1ca5baef5c7169c7da2ba7c009c908f20eca9358755d105faf0641fcf7f5cfd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a1ca5baef5c7169c7da2ba7c009c908f20eca9358755d105faf0641fcf7f5cfd/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a352edd24d84cc0fc288b36c7c48ce15dc2cd019d4e5bc396c8b242a82bc43c8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a352edd24d84cc0fc288b36c7c48ce15dc2cd019d4e5bc396c8b242a82bc43c8/userdata/shm major:0 minor:281 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a6d797fb42450ffe205b46017cda83bb21b1a4e8e3ea7d054661bcca72bb445b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a6d797fb42450ffe205b46017cda83bb21b1a4e8e3ea7d054661bcca72bb445b/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aeb7cb636cec5f7748fdc9f1f81dff156ae49a67936db2ad35181d08645b8831/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aeb7cb636cec5f7748fdc9f1f81dff156ae49a67936db2ad35181d08645b8831/userdata/shm major:0 minor:303 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c86ff0970a9bb85977512368549d35042c2899c8ad0c393ea8f2b77e40037c25/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c86ff0970a9bb85977512368549d35042c2899c8ad0c393ea8f2b77e40037c25/userdata/shm major:0 minor:277 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cd49119309dcb11aab8d8f869b8f8dc0fae222844bc784f4d5ae31d20b599ffb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cd49119309dcb11aab8d8f869b8f8dc0fae222844bc784f4d5ae31d20b599ffb/userdata/shm major:0 minor:332 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ce1ac0a68e3b02b8a5bec0ac63626c10ae12ae696f33bb32fd09b114e0270fbd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ce1ac0a68e3b02b8a5bec0ac63626c10ae12ae696f33bb32fd09b114e0270fbd/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d38c492d5386de0ea7cf31146fd539f68853b5d21dd5cd56353205012ca570f4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d38c492d5386de0ea7cf31146fd539f68853b5d21dd5cd56353205012ca570f4/userdata/shm major:0 minor:279 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e2439f6a602acc4de55b7b91cc20a90f0b8793364c7875738b90a8335828cecd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e2439f6a602acc4de55b7b91cc20a90f0b8793364c7875738b90a8335828cecd/userdata/shm major:0 minor:79 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e93a9a65d105becd00d0094f7a53873f2c021457989e5d1b00755c57d644e8e6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e93a9a65d105becd00d0094f7a53873f2c021457989e5d1b00755c57d644e8e6/userdata/shm major:0 minor:478 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/volumes/kubernetes.io~projected/kube-api-access-vpjcn:{mountpoint:/var/lib/kubelet/pods/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/volumes/kubernetes.io~projected/kube-api-access-vpjcn major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/web-config/alertmanager/9 major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/web-config/config-reloader/6 major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~empty-dir/config-out major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~projected/kube-api-access-4nhbx:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~projected/kube-api-access-4nhbx major:0 minor:317 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~projected/tls-assets major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/config-volume major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/web-config major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0adce241-e2df-40ec-8976-341e00ebc074/volumes/kubernetes.io~projected/kube-api-access-v7nwx:{mountpoint:/var/lib/kubelet/pods/0adce241-e2df-40ec-8976-341e00ebc074/volumes/kubernetes.io~projected/kube-api-access-v7nwx major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1062c6b4-6828-4e92-92a5-c78eccb1fe26/volumes/kubernetes.io~projected/kube-api-access-c4c5v:{mountpoint:/var/lib/kubelet/pods/1062c6b4-6828-4e92-92a5-c78eccb1fe26/volumes/kubernetes.io~projected/kube-api-access-c4c5v major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1062c6b4-6828-4e92-92a5-c78eccb1fe26/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/1062c6b4-6828-4e92-92a5-c78eccb1fe26/volumes/kubernetes.io~secret/metrics-certs major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9/volumes/kubernetes.io~projected/kube-api-access-5l97l:{mountpoint:/var/lib/kubelet/pods/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9/volumes/kubernetes.io~projected/kube-api-access-5l97l major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/162db930-0919-4366-ac55-b8e647cb714b/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/162db930-0919-4366-ac55-b8e647cb714b/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/162db930-0919-4366-ac55-b8e647cb714b/volumes/kubernetes.io~projected/kube-api-access-cqzpn:{mountpoint:/var/lib/kubelet/pods/162db930-0919-4366-ac55-b8e647cb714b/volumes/kubernetes.io~projected/kube-api-access-cqzpn major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/162db930-0919-4366-ac55-b8e647cb714b/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/162db930-0919-4366-ac55-b8e647cb714b/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a451455-21be-4f3a-a56d-8c4026bcf654/volumes/kubernetes.io~projected/kube-api-access-br287:{mountpoint:/var/lib/kubelet/pods/1a451455-21be-4f3a-a56d-8c4026bcf654/volumes/kubernetes.io~projected/kube-api-access-br287 major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a451455-21be-4f3a-a56d-8c4026bcf654/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/1a451455-21be-4f3a-a56d-8c4026bcf654/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:271 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a451455-21be-4f3a-a56d-8c4026bcf654/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/1a451455-21be-4f3a-a56d-8c4026bcf654/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:273 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:470 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~empty-dir/config-out major:0 minor:472 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~projected/kube-api-access-qgd46:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~projected/kube-api-access-qgd46 major:0 minor:475 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~projected/tls-assets major:0 minor:461 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/config major:0 minor:468 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:474 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:465 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:469 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:467 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:466 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:473 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:471 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/web-config major:0 minor:470 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2926b50e-e07d-445d-bbec-eab0005a59bf/volumes/kubernetes.io~projected/kube-api-access-8lh9t:{mountpoint:/var/lib/kubelet/pods/2926b50e-e07d-445d-bbec-eab0005a59bf/volumes/kubernetes.io~projected/kube-api-access-8lh9t major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/39289142-2079-4b45-ba09-8e06aed12276/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/39289142-2079-4b45-ba09-8e06aed12276/volumes/kubernetes.io~secret/agent-certs major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a9daaf5-4ea6-472e-9972-68cfeee5ae16/volumes/kubernetes.io~projected/kube-api-access-74s7t:{mountpoint:/var/lib/kubelet/pods/4a9daaf5-4ea6-472e-9972-68cfeee5ae16/volumes/kubernetes.io~projected/kube-api-access-74s7t major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a9daaf5-4ea6-472e-9972-68cfeee5ae16/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/4a9daaf5-4ea6-472e-9972-68cfeee5ae16/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4faa20e7-dd05-4416-9b9e-ed52b91d388d/volumes/kubernetes.io~projected/kube-api-access-srms2:{mountpoint:/var/lib/kubelet/pods/4faa20e7-dd05-4416-9b9e-ed52b91d388d/volumes/kubernetes.io~projected/kube-api-access-srms2 major:0 minor:302 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6214326d-31ce-4577-bd4b-7997c910a058/volumes/kubernetes.io~projected/kube-api-access-w4bf5:{mountpoint:/var/lib/kubelet/pods/6214326d-31ce-4577-bd4b-7997c910a058/volumes/kubernetes.io~projected/kube-api-access-w4bf5 major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6214326d-31ce-4577-bd4b-7997c910a058/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/6214326d-31ce-4577-bd4b-7997c910a058/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6214326d-31ce-4577-bd4b-7997c910a058/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/6214326d-31ce-4577-bd4b-7997c910a058/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/793c2518-45fa-46e0-9a23-766ae40d922b/volumes/kubernetes.io~projected/kube-api-access-7xhvx:{mountpoint:/var/lib/kubelet/pods/793c2518-45fa-46e0-9a23-766ae40d922b/volumes/kubernetes.io~projected/kube-api-access-7xhvx major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/793c2518-45fa-46e0-9a23-766ae40d922b/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/793c2518-45fa-46e0-9a23-766ae40d922b/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/853421d2-ef94-4cbd-b251-1f4d82bfb45e/volumes/kubernetes.io~projected/kube-api-access-2v5gg:{mountpoint:/var/lib/kubelet/pods/853421d2-ef94-4cbd-b251-1f4d82bfb45e/volumes/kubernetes.io~projected/kube-api-access-2v5gg major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~projected/kube-api-access-t8jdj:{mountpoint:/var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~projected/kube-api-access-t8jdj major:0 minor:385 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:379 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:384 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~projected/kube-api-access-hskxz:{mountpoint:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~projected/kube-api-access-hskxz major:0 minor:366 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:365 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:357 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b5c3d5df-2428-490f-925a-f67ef2e8c0b6/volumes/kubernetes.io~projected/kube-api-access-87s78:{mountpoint:/var/lib/kubelet/pods/b5c3d5df-2428-490f-925a-f67ef2e8c0b6/volumes/kubernetes.io~projected/kube-api-access-87s78 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b5c3d5df-2428-490f-925a-f67ef2e8c0b6/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/b5c3d5df-2428-490f-925a-f67ef2e8c0b6/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b8cc6455-76b2-40a2-aa43-5b516d85d543/volumes/kubernetes.io~projected/kube-api-access-wpbk9:{mountpoint:/var/lib/kubelet/pods/b8cc6455-76b2-40a2-aa43-5b516d85d543/volumes/kubernetes.io~projected/kube-api-access-wpbk9 major:0 minor:389 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b8cc6455-76b2-40a2-aa43-5b516d85d543/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/b8cc6455-76b2-40a2-aa43-5b516d85d543/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:388 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/volumes/kubernetes.io~projected/kube-api-access-jb64l:{mountpoint:/var/lib/kubelet/pods/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/volumes/kubernetes.io~projected/kube-api-access-jb64l major:0 minor:275 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:272 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:274 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c69d9666-d791-48fe-aad3-7a91f2ab2d2c/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/c69d9666-d791-48fe-aad3-7a91f2ab2d2c/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d/volumes/kubernetes.io~projected/kube-api-access-8kh5j:{mountpoint:/var/lib/kubelet/pods/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d/volumes/kubernetes.io~projected/kube-api-access-8kh5j major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56/volumes/kubernetes.io~projected/kube-api-access-jqs7p:{mountpoint:/var/lib/kubelet/pods/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56/volumes/kubernetes.io~projected/kube-api-access-jqs7p major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4471adf-01e8-47ae-8056-c670541a735c/volumes/kubernetes.io~projected/kube-api-access-bgt5q:{mountpoint:/var/lib/kubelet/pods/f4471adf-01e8-47ae-8056-c670541a735c/volumes/kubernetes.io~projected/kube-api-access-bgt5q major:0 minor:65 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/e813237a1989c199aa728a43c08ac01263fd99512db79c8da170e93cb70b7e24/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/3bfb2e7b7e623f4cf9e13ccd0fc8d57b4a0d32a41a62f6c42e764d3fc5283e94/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/85cf559c26e705a3363fbbf3993dce0e5928051c9c97858e2a1d4ac532c62486/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/5256ddd7ae8367940ba77e3efde4437216157770899d21d387b00828d7674c3b/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/afb3a93d22cf6c5ffd8d5e15bb7ca0af6522f9d3aa75b2f5e5b9df8cc8264895/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/289122d890864b645ece094b8c8d23fd606ec450f2d0103bc8722e97188130f4/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/1aaf3b7ec5bf99896bb3f62644849df97f0cd2075df24a453a6e86c9d43bf71a/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/9b77b6129bdfaa05895bab736d2ffb1b4ffd13d613db6dbffe50f2dea91bd940/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/f65c3ee185260df03c3aec305aa22fccbb25ea31cfce2959546ce3fb0452cf76/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/74ba7c21831c5dfbf60988214e24cfda7c8217f7b5049b88058595ddbc1387da/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/5b379d08d670d78bb925157678e79213e9ee26e6e094d4af423ab4ea6880f50b/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/76e59e26f58862fd1422c23a91190b17a2d4572b3b5881ad97ae241e3352c10d/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/e5cf183717dfb0de3cdabb3be51c5d879b3829915d09755d43b376acdb99d98b/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/a3d1fd436cccb17efc498c3f705021d48a5f7e5f8bf58efa327a9ef53a4e38f2/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/c67d42cc970f384b31b0d4616108e4a57cd0efd0f24c1c0f9523f06dc8b1e228/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/2d41b6cdeb68e926736d064be9732c2595267a9c85b014ae74b8f0d58e2baa93/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/5f2c79085837fee8a6bc5b0ef6ca39785bf63fed7f1e1faba6a7c21e07b5d53b/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/48a734d4c5f9588f3120ce2c93d87c7ec8a2dc04dd9e5ab5e3ae46248f8a9c7b/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/76190c12b40b5d1a64d6c44ba9fcd83fc6c264cabcae8075972f7c89936f9c62/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/19d5b3482e28bcfe5bf49680b9657f052b010c3a383f8afb47cc25d4cf94891b/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/2e7ab778b57fba648b4370e4c02d4a76884c7f55654ed73e47478a712592494b/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/6fc55fd2b7a0d4bc9f1e73ef54ea4945807dedc97d3e93e824929b4fc39c517f/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/54e346a2c74d931670a48755144d57bdc8accac04cc6f349281cd53b473c815f/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/1597478d35403eb4cd1e5e2cbb68f7f263b7d92bfd258f49739c4e1855c2c4a8/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/765d30a7bf5560efe094dd29c41696ca009fa742e4c402d17f77e9feef7d1415/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/dad6e0b6ed3ca82c11cae4205851c205618ef15a406481318b80f0dbd7aba67f/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/b5936d8d30d5166cb549966e846c7f3dd2a78bd6a3bb399e75b5c302ca5d65c1/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/f188f8c5bf365b287c9e83e90784167fc9758d233dc1329f48c227a836c8a212/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/3066e5de96898fbfcbc48f3ec8b6ffa5c3aae8ac3dc82934d4bf9e130bd73f19/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/c64d5762d8577862a273c0625770b2d48e5817d20ba971ee597ec3032aa2c63d/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/9ea982f42d7f2e33838c0cf765f1cc15e04b08405cee7a47a3ff7d217f7cc9ad/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/1b887f3fc467b12204f65ef7090068abac950d2969925e23e56d11f15e120465/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/1e14c5efb9e40f31f7b46e4c85c39c96515fb4da84879698f10e977e1d4a3861/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-283:{mountpoint:/var/lib/containers/storage/overlay/fcdc80c9093846d74c787c42e9478990d567ce5f1ded4edf1a96aae12157bf65/merged major:0 minor:283 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/dff4388550080c1ec205d3f81c27d797901b1ffe266d5383409a04b5396241b2/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/e24f38d4a73e9112b8cd2b719293b231f5ee4878ecf055f571ef02afc9161624/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/6b74b905ec92d6e2b1b1764bb9e77abec20bfabbf0cde33bfcaa46f222cbd7f6/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-290:{mountpoint:/var/lib/containers/storage/overlay/e82b65d93a898007ea9b80519f4fdecf3684f763a1a9d9446b389ba571b2ec41/merged major:0 minor:290 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/91134daec7b29376163e227ea2628662ae23428b0df77b73440cb876c7611d0d/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-311:{mountpoint:/var/lib/containers/storage/overlay/ed6c60d0ed4f0396413437a6d4c13ad53fa8764e8f04c35c30953037cd65fe95/merged major:0 minor:311 fsType:overlay blockSize:0} overlay_0-313:{mountpoint:/var/lib/containers/storage/overlay/236d8f885846ae4a331ddcd16863bdd2383b7d22d8eac8ecbefe6751a45b3d30/merged major:0 minor:313 fsType:overlay blockSize:0} overlay_0-315:{mountpoint:/var/lib/containers/storage/overlay/0816aa75b319b8e09365f092f168bafed3b1ebf679eb6cd0f15c8850ded032f2/merged major:0 minor:315 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/35f9858517cbd359c4f6b3ccdd17e83fdb24ec89fbbf57df4c15bd61d89d40d0/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-337:{mountpoint:/var/lib/containers/storage/overlay/ea17ee60e4894b4e0ca8cc28c9bfc68b0ed8e14c536494b33f68b3cd6942727a/merged major:0 minor:337 fsType:overlay blockSize:0} overlay_0-339:{mountpoint:/var/lib/containers/storage/overlay/8c3fee560b1a5a72e95388d43afbfaa1ffbe271051dcd543cdd430367139b0d5/merged major:0 minor:339 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/cee34a70673de10abac7e8bfd03f2d4ec7ea8edc7333bdd6fe9a93909f39540b/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/74c6e9c526767b4ae303f452f0d7b75ed71307d63b87a646d2372408dff5c3c6/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/2cab15f512b1edf5275a9356208f3e2abcccb482a2cbbb5577b55e798c8e5358/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-369:{mountpoint:/var/lib/containers/storage/overlay/ad4fdbc12ba142badc5e1eb5cf25c570081bf5aed851f21df78d91a9e2c8eb75/merged major:0 minor:369 fsType:overlay blockSize:0} overlay_0-371:{mountpoint:/var/lib/containers/storage/overlay/93293780dac65309c78dfa3aa0059e9985b76299c60eaa84347040adb810b543/merged major:0 minor:371 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/304e815085c48ab2baf5775b5a70cfc6213ef68d54234b444bc15224b3c939f3/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-392:{mountpoint:/var/lib/containers/storage/overlay/5c20bf387b94226588ea146c9ddc2fa7af1d3fc34511548a96a606934d81d6c9/merged major:0 minor:392 fsType:overlay blockSize:0} overlay_0-394:{mountpoint:/var/lib/containers/storage/overlay/7b7c9ad176d3d6c9e9b4641a6e527d32c6a82f4b9f5c1fbc0e9e12ee48a5448a/merged major:0 minor:394 fsType:overlay blockSize:0} overlay_0-396:{mountpoint:/var/lib/containers/storage/overlay/329b80d87fdf115658e15b5c2fa4c788625d50d1824b3582a9685c3382e8d844/merged major:0 minor:396 fsType:overlay blockSize:0} overlay_0-398:{mountpoint:/var/lib/containers/storage/overlay/11b99cca9334fda9295444299def88645dd7c651a2630e0aab92a9ed84527067/merged major:0 minor:398 fsType:overlay blockSize:0} overlay_0-400:{mountpoint:/var/lib/containers/storage/overlay/21df4a88821a55b632dade8ae2741eec88af9e26e6cb25a2c9215efe7ff59883/merged major:0 minor:400 fsType:overlay blockSize:0} overlay_0-410:{mountpoint:/var/lib/containers/storage/overlay/2369a355906986d5fb2cd54f92416a61d44805058e869774b2975a5289f870e3/merged major:0 minor:410 fsType:overlay blockSize:0} overlay_0-412:{mountpoint:/var/lib/containers/storage/overlay/99e9cda34c14464c092f4975078b0b6cf6a2cb1bf819836419aa649dc818e183/merged major:0 minor:412 fsType:overlay blockSize:0} overlay_0-420:{mountpoint:/var/lib/containers/storage/overlay/e2c08de7cac0f07f815592826dd0bec13b81b8e8b00b61cdbbf298cdbf9f22a2/merged major:0 minor:420 fsType:overlay blockSize:0} overlay_0-422:{mountpoint:/var/lib/containers/storage/overlay/5bb48959620c5216e2a1f7adef7ff2b49046ae3983572dc663dccfa3c9df4549/merged major:0 minor:422 fsType:overlay blockSize:0} overlay_0-423:{mountpoint:/var/lib/containers/storage/overlay/9d47998166208dd5251f4854574d3844f31a1c44531e11999bf49a9225ad9150/merged major:0 minor:423 fsType:overlay blockSize:0} overlay_0-435:{mountpoint:/var/lib/containers/storage/overlay/29243b1e60fffa577898502d20d8fa7194f2257a105fff8759fec8b6d5b09505/merged major:0 minor:435 fsType:overlay blockSize:0} overlay_0-440:{mountpoint:/var/lib/containers/storage/overlay/d028c8322424e0430f6a8508fa624c4448e49bd178cfe6a88604d03b1d989dc6/merged major:0 minor:440 fsType:overlay blockSize:0} overlay_0-446:{mountpoint:/var/lib/containers/storage/overlay/13b36913c898e593852a6063ec85188b3218c48f4662cec85f5e20865d64e4ba/merged major:0 minor:446 fsType:overlay blockSize:0} overlay_0-450:{mountpoint:/var/lib/containers/storage/overlay/905949297c7c1f921b031305c9d3ea064d56ec2dfe0c262bc6e39d253542bd51/merged major:0 minor:450 fsType:overlay blockSize:0} overlay_0-457:{mountpoint:/var/lib/containers/storage/overlay/0c467ba26f11310264b858742d5ca352bab693750eab27dc88fca7a3830b0748/merged major:0 minor:457 fsType:overlay blockSize:0} overlay_0-476:{mountpoint:/var/lib/containers/storage/overlay/bdd181825038d02f0c72a5f1f354ec1d92bbbb36acdcce80bc514f1bcc4b8ebd/merged major:0 minor:476 fsType:overlay blockSize:0} overlay_0-480:{mountpoint:/var/lib/containers/storage/overlay/e1894e24df122ce71bce54917993118da453e5971f03db7fc443aaa55148ec63/merged major:0 minor:480 fsType:overlay blockSize:0} overlay_0-482:{mountpoint:/var/lib/containers/storage/overlay/4555bf9d60755a60453f78ca77325add232e574dc11323867762632ede60be19/merged major:0 minor:482 fsType:overlay blockSize:0} overlay_0-492:{mountpoint:/var/lib/containers/storage/overlay/d51a2ae7bb87373809bb227e5581c2afe666f26d3f47500ce44c4178ad16cb4f/merged major:0 minor:492 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/6ce0a8660b48b79561d73169ef1b2d5a7add6515fe5c3290d8fe144b8ebc7091/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/4ddae071f7fff4c69518a0d5b4aa2e0358847b07ffe0d2504f8f8c2f89d6aef2/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/42f2992ff730199e7ec77aa4846abb4663e58b50d7af103aa33d15d950227b71/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/a5cf1e968b77468862c181abc3332b5b159e05929dffa408f1557fd42e835b97/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/b6a15ad4d4c6045ec96d5ba1a4ca72797c89cf6ca0ffd1210dd7a7d93e16bbfe/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-81:{mountpoint:/var/lib/containers/storage/overlay/ba26638ced0a109abf72fef25c2e5942c8de9617699986d02c38606500ef50ff/merged major:0 minor:81 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/5a616f108cf083d89c3b27168da0a3cf0bba98e09bc7ecf3fd80440c65e686d9/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/0d7c042d580a7b5d6090ba5d8aaa63f0c1ae48dd55ad045af7a343bef094db52/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/5ce6e4d3bfe0c6821fdaec11e9aec64855e7b288c5cf034ef38bf01d0de9be8e/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-97:{mountpoint:/var/lib/containers/storage/overlay/e6bec813769e03e5d15cd1a14d4def2fd495112b94427d3d52424a312d1fc654/merged major:0 minor:97 fsType:overlay blockSize:0}]
Oct 30 12:57:42.770925 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.770419    5781 manager.go:217] Machine: {Timestamp:2026-10-30 12:57:42.76991936 +0000 UTC m=+0.067136895 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102942 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20920f0b2a032fd856ae91ba40243c SystemUUID:ec20920f-0b2a-032f-d856-ae91ba40243c BootID:c055694b-75f1-475b-a05a-5599528bee6b Filesystems:[{Device:/run/containers/storage/overlay-containers/ce1ac0a68e3b02b8a5bec0ac63626c10ae12ae696f33bb32fd09b114e0270fbd/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-81 DeviceMajor:0 DeviceMinor:81 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:274 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/62bb2d3a02381a4358ccbf6e514547ed66f9256697136170e2ee1af5d74ab0a7/userdata/shm DeviceMajor:0 DeviceMinor:307 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:323 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:329 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-410 DeviceMajor:0 DeviceMinor:410 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/71446ba892b17259700ac5db04ae23a27f073e8569b9bada878a8427adf79bc9/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-371 DeviceMajor:0 DeviceMinor:371 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:328 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:364 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6214326d-31ce-4577-bd4b-7997c910a058/volumes/kubernetes.io~projected/kube-api-access-w4bf5 DeviceMajor:0 DeviceMinor:270 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-313 DeviceMajor:0 DeviceMinor:313 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~projected/kube-api-access-t8jdj DeviceMajor:0 DeviceMinor:385 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-394 DeviceMajor:0 DeviceMinor:394 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a352edd24d84cc0fc288b36c7c48ce15dc2cd019d4e5bc396c8b242a82bc43c8/userdata/shm DeviceMajor:0 DeviceMinor:281 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:322 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:362 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-422 DeviceMajor:0 DeviceMinor:422 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-457 DeviceMajor:0 DeviceMinor:457 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6214326d-31ce-4577-bd4b-7997c910a058/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:265 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:272 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-290 DeviceMajor:0 DeviceMinor:290 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:361 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f4471adf-01e8-47ae-8056-c670541a735c/volumes/kubernetes.io~projected/kube-api-access-bgt5q DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0f5ddbb728fbacf85b52607447cfa8da1ebc445a470d9cb53306a0bf051f5009/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e2439f6a602acc4de55b7b91cc20a90f0b8793364c7875738b90a8335828cecd/userdata/shm DeviceMajor:0 DeviceMinor:79 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8321f08e19f4910820e6872cfb6f88739d17c098e9875cfa591d843b3995946c/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:329 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/853421d2-ef94-4cbd-b251-1f4d82bfb45e/volumes/kubernetes.io~projected/kube-api-access-2v5gg DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/162db930-0919-4366-ac55-b8e647cb714b/volumes/kubernetes.io~projected/kube-api-access-cqzpn DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1c4544afd27f657628273ca69886f70932c2fa5a4ba34123ba5e5ce43203e091/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:324 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:326 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:365 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a6d797fb42450ffe205b46017cda83bb21b1a4e8e3ea7d054661bcca72bb445b/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-339 DeviceMajor:0 DeviceMinor:339 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b8cc6455-76b2-40a2-aa43-5b516d85d543/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:388 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b8cc6455-76b2-40a2-aa43-5b516d85d543/volumes/kubernetes.io~projected/kube-api-access-wpbk9 DeviceMajor:0 DeviceMinor:389 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:467 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:470 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8ce437055d0458893d41311eb3c61791c6d93b3f83bf6e99a5ba01816b252668/userdata/shm DeviceMajor:0 DeviceMinor:233 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-446 DeviceMajor:0 DeviceMinor:446 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-476 DeviceMajor:0 DeviceMinor:476 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-435 DeviceMajor:0 DeviceMinor:435 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-450 DeviceMajor:0 DeviceMinor:450 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:472 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:327 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:363 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:383 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:325 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:473 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:474 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:379 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:469 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0adce241-e2df-40ec-8976-341e00ebc074/volumes/kubernetes.io~projected/kube-api-access-v7nwx DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a9daaf5-4ea6-472e-9972-68cfeee5ae16/volumes/kubernetes.io~projected/kube-api-access-74s7t DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:229 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6214326d-31ce-4577-bd4b-7997c910a058/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:269 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-423 DeviceMajor:0 DeviceMinor:423 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4f0d02a497aea0a09e35155819127d1154dc15d08de78b7cf3be97c45a3ae79c/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1a451455-21be-4f3a-a56d-8c4026bcf654/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:273 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:321 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:357 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/99f8c0d1-d514-488c-9401-b745e93ad9ac/volumes/kubernetes.io~projected/kube-api-access-hskxz DeviceMajor:0 DeviceMinor:366 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b5c3d5df-2428-490f-925a-f67ef2e8c0b6/volumes/kubernetes.io~projected/kube-api-access-87s78 DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/60fa67f9cce7dfdda00d96d1f2f07818fb787f1d8e8951492ba1b524941495ef/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4faa20e7-dd05-4416-9b9e-ed52b91d388d/volumes/kubernetes.io~projected/kube-api-access-srms2 DeviceMajor:0 DeviceMinor:302 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/88dac2fc3b5fcc3a3c2de4bb891256e0babeeaa3046db943f0c7ebb0358a3228/userdata/shm DeviceMajor:0 DeviceMinor:304 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/88f01f9c-02af-4e17-9a80-7f769221c703/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:384 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-283 DeviceMajor:0 DeviceMinor:283 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:329 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:466 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-492 DeviceMajor:0 DeviceMinor:492 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a1ca5baef5c7169c7da2ba7c009c908f20eca9358755d105faf0641fcf7f5cfd/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-315 DeviceMajor:0 DeviceMinor:315 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:325 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cd49119309dcb11aab8d8f869b8f8dc0fae222844bc784f4d5ae31d20b599ffb/userdata/shm DeviceMajor:0 DeviceMinor:332 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:461 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-97 DeviceMajor:0 DeviceMinor:97 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/volumes/kubernetes.io~projected/kube-api-access-vpjcn DeviceMajor:0 DeviceMinor:230 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-480 DeviceMajor:0 DeviceMinor:480 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1062c6b4-6828-4e92-92a5-c78eccb1fe26/volumes/kubernetes.io~projected/kube-api-access-c4c5v DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/162db930-0919-4366-ac55-b8e647cb714b/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/1a451455-21be-4f3a-a56d-8c4026bcf654/volumes/kubernetes.io~projected/kube-api-access-br287 DeviceMajor:0 DeviceMinor:276 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:465 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/39289142-2079-4b45-ba09-8e06aed12276/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7b63fe8b9bff669d6d61a0846f7fa93f8ac602b5b3f2f2b049b537fa5b2efbf2/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-337 DeviceMajor:0 DeviceMinor:337 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-396 DeviceMajor:0 DeviceMinor:396 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-440 DeviceMajor:0 DeviceMinor:440 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:471 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~projected/kube-api-access-qgd46 DeviceMajor:0 DeviceMinor:475 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/162db930-0919-4366-ac55-b8e647cb714b/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/793c2518-45fa-46e0-9a23-766ae40d922b/volumes/kubernetes.io~projected/kube-api-access-7xhvx DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c86ff0970a9bb85977512368549d35042c2899c8ad0c393ea8f2b77e40037c25/userdata/shm DeviceMajor:0 DeviceMinor:277 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-369 DeviceMajor:0 DeviceMinor:369 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:470 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/914daae99f98a21338ff4a5f32aba5ef1c8376b8411f53722835adbe3af5a601/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2926b50e-e07d-445d-bbec-eab0005a59bf/volumes/kubernetes.io~projected/kube-api-access-8lh9t DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-392 DeviceMajor:0 DeviceMinor:392 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-412 DeviceMajor:0 DeviceMinor:412 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:225 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-400 DeviceMajor:0 DeviceMinor:400 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b5c3d5df-2428-490f-925a-f67ef2e8c0b6/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9a9f8ba48bd42ba5051e5e2d12b62c1b47447d2b9c2eb5eca82a2e2999e4a534/userdata/shm DeviceMajor:0 DeviceMinor:367 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56/volumes/kubernetes.io~projected/kube-api-access-jqs7p DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d/volumes/kubernetes.io~projected/kube-api-access-8kh5j DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/793c2518-45fa-46e0-9a23-766ae40d922b/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1a451455-21be-4f3a-a56d-8c4026bcf654/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:271 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-311 DeviceMajor:0 DeviceMinor:311 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:468 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a9daaf5-4ea6-472e-9972-68cfeee5ae16/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-398 DeviceMajor:0 DeviceMinor:398 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e93a9a65d105becd00d0094f7a53873f2c021457989e5d1b00755c57d644e8e6/userdata/shm DeviceMajor:0 DeviceMinor:478 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-482 DeviceMajor:0 DeviceMinor:482 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1062c6b4-6828-4e92-92a5-c78eccb1fe26/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:297 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aeb7cb636cec5f7748fdc9f1f81dff156ae49a67936db2ad35181d08645b8831/userdata/shm DeviceMajor:0 DeviceMinor:303 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/26b4d4e3bf0dc818e862ac74b01999e2ea17b713606006528313d9f86714c102/userdata/shm DeviceMajor:0 DeviceMinor:386 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9/volumes/kubernetes.io~projected/kube-api-access-5l97l DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/volumes/kubernetes.io~projected/kube-api-access-jb64l DeviceMajor:0 DeviceMinor:275 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9a33816ef2cd30b219a540c24f03ef7342a59fb9412e1901e2c6f606ab997294/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d38c492d5386de0ea7cf31146fd539f68853b5d21dd5cd56353205012ca570f4/userdata/shm DeviceMajor:0 DeviceMinor:279 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:329 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7050d3799203f75c4bc749324f41dfa9d14fb188f53510c4ec1af62dbdc7287a/userdata/shm DeviceMajor:0 DeviceMinor:390 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-420 DeviceMajor:0 DeviceMinor:420 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c69d9666-d791-48fe-aad3-7a91f2ab2d2c/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:301 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes/kubernetes.io~projected/kube-api-access-4nhbx DeviceMajor:0 DeviceMinor:317 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:26b4d4e3bf0dc81 MacAddress:ca:1a:de:e1:a7:62 Speed:10000 Mtu:8901} {Name:4f0d02a497aea0a MacAddress:86:3e:97:36:71:72 Speed:10000 Mtu:8901} {Name:60fa67f9cce7dfd MacAddress:92:1a:8f:d2:e7:5a Speed:10000 Mtu:8901} {Name:62bb2d3a02381a4 MacAddress:52:eb:ad:38:d4:a5 Speed:10000 Mtu:8901} {Name:7050d3799203f75 MacAddress:a6:97:79:a1:49:7c Speed:10000 Mtu:8901} {Name:8321f08e19f4910 MacAddress:5a:3e:aa:2f:8c:c1 Speed:10000 Mtu:8901} {Name:88dac2fc3b5fcc3 MacAddress:1a:27:25:fe:d8:1f Speed:10000 Mtu:8901} {Name:8ce437055d04588 MacAddress:0e:ad:18:b3:ff:a4 Speed:10000 Mtu:8901} {Name:9a9f8ba48bd42ba MacAddress:f6:34:0b:cd:f8:65 Speed:10000 Mtu:8901} {Name:aeb7cb636cec5f7 MacAddress:82:a5:ee:b1:0e:a8 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c9:bd:90:57 Speed:0 Mtu:9001} {Name:br-int MacAddress:9a:84:37:fa:ab:98 Speed:0 Mtu:8901} {Name:c86ff0970a9bb85 MacAddress:0a:f0:27:bf:0f:03 Speed:10000 Mtu:8901} {Name:cd49119309dcb11 MacAddress:b6:46:39:de:4b:e9 Speed:10000 Mtu:8901} {Name:d38c492d5386de0 MacAddress:36:ab:aa:fc:2c:70 Speed:10000 Mtu:8901} {Name:e93a9a65d105bec MacAddress:ce:09:10:44:f0:5d Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c9:bd:90:57 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:66:8b:b9:72:f7:54 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:06:7b:33:07:c3:0c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 12:57:42.771170 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.770930    5781 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 12:57:42.771170 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.770982    5781 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 12:57:42.771267 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771247    5781 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 12:57:42.771436 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771268    5781 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-180.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 12:57:42.771482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771438    5781 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 12:57:42.771482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771447    5781 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 12:57:42.771482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771461    5781 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:57:42.771482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771474    5781 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:57:42.771583 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771573    5781 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 12:57:42.771743 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771733    5781 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 12:57:42.771853 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771844    5781 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 12:57:42.771888 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771881    5781 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 12:57:42.771917 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771893    5781 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 12:57:42.771917 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771903    5781 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 12:57:42.771917 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771912    5781 kubelet.go:405] "Adding apiserver pod source"
Oct 30 12:57:42.772002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.771920    5781 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 12:57:42.772654 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.772636    5781 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 12:57:42.772654 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.772649    5781 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:57:42.773111 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.773097    5781 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 12:57:42.792494 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792475    5781 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 12:57:42.792494 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792489    5781 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:57:42.792606 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792588    5781 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 12:57:42.792784 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792772    5781 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 12:57:42.792817 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792802    5781 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 12:57:42.792902 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792894    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 12:57:42.792931 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792908    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 12:57:42.792931 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792926    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 12:57:42.792981 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792933    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 12:57:42.792981 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792939    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 12:57:42.792981 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792947    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 12:57:42.792981 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792952    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 12:57:42.792981 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792958    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 12:57:42.792981 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792964    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 12:57:42.792981 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792970    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 12:57:42.792981 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792983    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 12:57:42.793186 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.792993    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 12:57:42.793186 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.793016    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 12:57:42.793186 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.793022    5781 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 12:57:42.796446 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.796432    5781 server.go:1263] "Started kubelet"
Oct 30 12:57:42.796533 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.796503    5781 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 12:57:42.796533 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.796616    5781 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 12:57:42.799680 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.797706    5781 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 12:57:42.797722 ip-10-0-130-180 systemd[1]: Started Kubernetes Kubelet.
Oct 30 12:57:42.800020 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.799972    5781 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 12:57:42.800432 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.800411    5781 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 12:57:42.803659 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.803636    5781 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 12:57:42.803659 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.803657    5781 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 12:57:42.806457 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.803735    5781 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 12:57:42.806457 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.803746    5781 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 12:57:42.806457 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:57:42.803802    5781 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-180.ec2.internal\" not found"
Oct 30 12:57:42.806457 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.803809    5781 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 12:57:42.806457 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.803714    5781 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:51:30 +0000 UTC" deadline="2028-06-25 08:57:48.62137135 +0000 UTC"
Oct 30 12:57:42.806457 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.803850    5781 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14492h0m5.817525154s"
Oct 30 12:57:42.806694 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806661    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4" volumeName="kubernetes.io/empty-dir/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-volume-directive-shadow" seLinuxMountContext=""
Oct 30 12:57:42.806732 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806700    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config" seLinuxMountContext=""
Oct 30 12:57:42.806732 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806714    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0adce241-e2df-40ec-8976-341e00ebc074" volumeName="kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-multus-daemon-config" seLinuxMountContext=""
Oct 30 12:57:42.806799 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806735    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1062c6b4-6828-4e92-92a5-c78eccb1fe26" volumeName="kubernetes.io/projected/1062c6b4-6828-4e92-92a5-c78eccb1fe26-kube-api-access-c4c5v" seLinuxMountContext=""
Oct 30 12:57:42.806799 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806748    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a451455-21be-4f3a-a56d-8c4026bcf654" volumeName="kubernetes.io/projected/1a451455-21be-4f3a-a56d-8c4026bcf654-kube-api-access-br287" seLinuxMountContext=""
Oct 30 12:57:42.806799 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806761    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets" seLinuxMountContext=""
Oct 30 12:57:42.806799 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806773    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806786    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806822    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a9daaf5-4ea6-472e-9972-68cfeee5ae16" volumeName="kubernetes.io/configmap/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806836    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa" volumeName="kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806851    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806860    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806872    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a451455-21be-4f3a-a56d-8c4026bcf654" volumeName="kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806884    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a451455-21be-4f3a-a56d-8c4026bcf654" volumeName="kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806900    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806912    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="88f01f9c-02af-4e17-9a80-7f769221c703" volumeName="kubernetes.io/empty-dir/88f01f9c-02af-4e17-9a80-7f769221c703-audit-log" seLinuxMountContext=""
Oct 30 12:57:42.806937 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806924    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="88f01f9c-02af-4e17-9a80-7f769221c703" volumeName="kubernetes.io/projected/88f01f9c-02af-4e17-9a80-7f769221c703-kube-api-access-t8jdj" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806937    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806964    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99f8c0d1-d514-488c-9401-b745e93ad9ac" volumeName="kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-grpc-tls" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806973    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8cc6455-76b2-40a2-aa43-5b516d85d543" volumeName="kubernetes.io/secret/b8cc6455-76b2-40a2-aa43-5b516d85d543-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806983    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4471adf-01e8-47ae-8056-c670541a735c" volumeName="kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.806997    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a451455-21be-4f3a-a56d-8c4026bcf654" volumeName="kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-metrics-client-ca" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807009    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="793c2518-45fa-46e0-9a23-766ae40d922b" volumeName="kubernetes.io/empty-dir/793c2518-45fa-46e0-9a23-766ae40d922b-tmp-dir" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807021    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="88f01f9c-02af-4e17-9a80-7f769221c703" volumeName="kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807034    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99f8c0d1-d514-488c-9401-b745e93ad9ac" volumeName="kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807046    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4" volumeName="kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807059    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4" volumeName="kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807072    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4" volumeName="kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807090    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d" volumeName="kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-tmp" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807103    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162db930-0919-4366-ac55-b8e647cb714b" volumeName="kubernetes.io/secret/162db930-0919-4366-ac55-b8e647cb714b-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807115    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807127    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807140    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807154    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2926b50e-e07d-445d-bbec-eab0005a59bf" volumeName="kubernetes.io/projected/2926b50e-e07d-445d-bbec-eab0005a59bf-kube-api-access-8lh9t" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807179    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6214326d-31ce-4577-bd4b-7997c910a058" volumeName="kubernetes.io/configmap/6214326d-31ce-4577-bd4b-7997c910a058-metrics-client-ca" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807192    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="88f01f9c-02af-4e17-9a80-7f769221c703" volumeName="kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807204    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807218    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config" seLinuxMountContext=""
Oct 30 12:57:42.807228 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807231    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807244    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed7b27fb-3f62-473c-89d0-ea4ced2bbe56" volumeName="kubernetes.io/projected/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kube-api-access-jqs7p" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807257    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4471adf-01e8-47ae-8056-c670541a735c" volumeName="kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-binary-copy" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807269    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807282    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807342    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0adce241-e2df-40ec-8976-341e00ebc074" volumeName="kubernetes.io/projected/0adce241-e2df-40ec-8976-341e00ebc074-kube-api-access-v7nwx" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807350    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1062c6b4-6828-4e92-92a5-c78eccb1fe26" volumeName="kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807359    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807367    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6214326d-31ce-4577-bd4b-7997c910a058" volumeName="kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807373    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d" volumeName="kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-tuned" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807382    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807394    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807401    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807410    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4faa20e7-dd05-4416-9b9e-ed52b91d388d" volumeName="kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807417    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="793c2518-45fa-46e0-9a23-766ae40d922b" volumeName="kubernetes.io/projected/793c2518-45fa-46e0-9a23-766ae40d922b-kube-api-access-7xhvx" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807424    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99f8c0d1-d514-488c-9401-b745e93ad9ac" volumeName="kubernetes.io/configmap/99f8c0d1-d514-488c-9401-b745e93ad9ac-metrics-client-ca" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807431    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4" volumeName="kubernetes.io/projected/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-api-access-jb64l" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807440    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4471adf-01e8-47ae-8056-c670541a735c" volumeName="kubernetes.io/projected/f4471adf-01e8-47ae-8056-c670541a735c-kube-api-access-bgt5q" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807457    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807465    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807472    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39289142-2079-4b45-ba09-8e06aed12276" volumeName="kubernetes.io/configmap/39289142-2079-4b45-ba09-8e06aed12276-konnectivity-ca" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807479    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="88f01f9c-02af-4e17-9a80-7f769221c703" volumeName="kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807486    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99f8c0d1-d514-488c-9401-b745e93ad9ac" volumeName="kubernetes.io/empty-dir/99f8c0d1-d514-488c-9401-b745e93ad9ac-tmp" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807493    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99f8c0d1-d514-488c-9401-b745e93ad9ac" volumeName="kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807500    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807508    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a451455-21be-4f3a-a56d-8c4026bcf654" volumeName="kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-tls" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807515    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b5c3d5df-2428-490f-925a-f67ef2e8c0b6" volumeName="kubernetes.io/secret/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-cert" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807522    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c69d9666-d791-48fe-aad3-7a91f2ab2d2c" volumeName="kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807529    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d" volumeName="kubernetes.io/projected/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-kube-api-access-8kh5j" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807536    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4471adf-01e8-47ae-8056-c670541a735c" volumeName="kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807544    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807551    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807558    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807566    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="39289142-2079-4b45-ba09-8e06aed12276" volumeName="kubernetes.io/secret/39289142-2079-4b45-ba09-8e06aed12276-agent-certs" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807574    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a9daaf5-4ea6-472e-9972-68cfeee5ae16" volumeName="kubernetes.io/secret/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807583    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b8cc6455-76b2-40a2-aa43-5b516d85d543" volumeName="kubernetes.io/projected/b8cc6455-76b2-40a2-aa43-5b516d85d543-kube-api-access-wpbk9" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807590    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2926b50e-e07d-445d-bbec-eab0005a59bf" volumeName="kubernetes.io/configmap/2926b50e-e07d-445d-bbec-eab0005a59bf-serviceca" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807598    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa" volumeName="kubernetes.io/projected/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-kube-api-access-vpjcn" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807605    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807612    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15d363e1-d64d-4ab4-a134-08ef9b8dd9f9" volumeName="kubernetes.io/projected/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-kube-api-access-5l97l" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807620    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162db930-0919-4366-ac55-b8e647cb714b" volumeName="kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807627    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162db930-0919-4366-ac55-b8e647cb714b" volumeName="kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-env-overrides" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807634    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162db930-0919-4366-ac55-b8e647cb714b" volumeName="kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-config" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807642    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807650    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa" volumeName="kubernetes.io/configmap/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-metrics-client-ca" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807656    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807664    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a9daaf5-4ea6-472e-9972-68cfeee5ae16" volumeName="kubernetes.io/empty-dir/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-data-volume" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807673    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6214326d-31ce-4577-bd4b-7997c910a058" volumeName="kubernetes.io/projected/6214326d-31ce-4577-bd4b-7997c910a058-kube-api-access-w4bf5" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807680    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6214326d-31ce-4577-bd4b-7997c910a058" volumeName="kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807688    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="853421d2-ef94-4cbd-b251-1f4d82bfb45e" volumeName="kubernetes.io/empty-dir/853421d2-ef94-4cbd-b251-1f4d82bfb45e-tmp-dir" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807695    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="88f01f9c-02af-4e17-9a80-7f769221c703" volumeName="kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-client-ca-bundle" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807702    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99f8c0d1-d514-488c-9401-b745e93ad9ac" volumeName="kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807709    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0adce241-e2df-40ec-8976-341e00ebc074" volumeName="kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-cni-binary-copy" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807717    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15d363e1-d64d-4ab4-a134-08ef9b8dd9f9" volumeName="kubernetes.io/configmap/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-iptables-alerter-script" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807723    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162db930-0919-4366-ac55-b8e647cb714b" volumeName="kubernetes.io/projected/162db930-0919-4366-ac55-b8e647cb714b-kube-api-access-cqzpn" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807731    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807739    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a9daaf5-4ea6-472e-9972-68cfeee5ae16" volumeName="kubernetes.io/projected/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-api-access-74s7t" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807751    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="88f01f9c-02af-4e17-9a80-7f769221c703" volumeName="kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807758    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99f8c0d1-d514-488c-9401-b745e93ad9ac" volumeName="kubernetes.io/projected/99f8c0d1-d514-488c-9401-b745e93ad9ac-kube-api-access-hskxz" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807765    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b5c3d5df-2428-490f-925a-f67ef2e8c0b6" volumeName="kubernetes.io/projected/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-kube-api-access-87s78" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807772    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0855ce73-dcfd-4398-b658-301a73ea990d" volumeName="kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807780    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a451455-21be-4f3a-a56d-8c4026bcf654" volumeName="kubernetes.io/empty-dir/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-textfile" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807787    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="853421d2-ef94-4cbd-b251-1f4d82bfb45e" volumeName="kubernetes.io/projected/853421d2-ef94-4cbd-b251-1f4d82bfb45e-kube-api-access-2v5gg" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807794    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99f8c0d1-d514-488c-9401-b745e93ad9ac" volumeName="kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807801    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa" volumeName="kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807808    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807816    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="221f1eb2-45dc-4f3e-b7d3-8487549100b4" volumeName="kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807824    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="793c2518-45fa-46e0-9a23-766ae40d922b" volumeName="kubernetes.io/configmap/793c2518-45fa-46e0-9a23-766ae40d922b-config-volume" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807832    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="793c2518-45fa-46e0-9a23-766ae40d922b" volumeName="kubernetes.io/secret/793c2518-45fa-46e0-9a23-766ae40d922b-metrics-tls" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807839    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99f8c0d1-d514-488c-9401-b745e93ad9ac" volumeName="kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807846    5781 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4" volumeName="kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-metrics-client-ca" seLinuxMountContext=""
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807854    5781 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 12:57:42.807827 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.807859    5781 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 12:57:42.810563 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.808957    5781 factory.go:55] Registering systemd factory
Oct 30 12:57:42.810563 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.808969    5781 factory.go:223] Registration of the systemd container factory successfully
Oct 30 12:57:42.810563 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.809183    5781 factory.go:153] Registering CRI-O factory
Oct 30 12:57:42.810563 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.809197    5781 factory.go:223] Registration of the crio container factory successfully
Oct 30 12:57:42.810563 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.809246    5781 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 12:57:42.810563 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.809270    5781 factory.go:103] Registering Raw factory
Oct 30 12:57:42.810563 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.809302    5781 manager.go:1199] Started watching for new ooms in manager
Oct 30 12:57:42.810563 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.809671    5781 manager.go:319] Starting recovery of all containers
Oct 30 12:57:42.830264 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.829795    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 12:57:42.831375 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.831325    5781 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 12:57:42.833207 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.833177    5781 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 12:57:42.833207 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.833203    5781 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 12:57:42.833454 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.833352    5781 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 12:57:42.833454 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:57:42.833409    5781 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 12:57:42.836866 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.836845    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 12:57:42.842231 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.842207    5781 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:57:42.843492 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.843475    5781 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:57:42.843639 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.843621    5781 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 12:57:42.854988 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.854963    5781 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:57:42.855095 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.855034    5781 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:57:42.858819 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.858788    5781 generic.go:356] "Generic (PLEG): container finished" podID="0855ce73-dcfd-4398-b658-301a73ea990d" containerID="7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f" exitCode=0
Oct 30 12:57:42.860963 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.860936    5781 generic.go:356] "Generic (PLEG): container finished" podID="1a451455-21be-4f3a-a56d-8c4026bcf654" containerID="524e08660622872a417606f8c83934b6effa70e48378912d8b2c04b49c516578" exitCode=0
Oct 30 12:57:42.871256 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.871230    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 12:57:42.871689 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.871658    5781 generic.go:356] "Generic (PLEG): container finished" podID="162db930-0919-4366-ac55-b8e647cb714b" containerID="15c51bdd0d9fa7a74f73c940004eb7c1c0f5d5888e9e51fef64d7e2da121ba14" exitCode=1
Oct 30 12:57:42.873933 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.873911    5781 generic.go:356] "Generic (PLEG): container finished" podID="9b3f18c352b4e52d3d8a287983d93c2b" containerID="543935082c87ae7e02833bd8ed70e93deac60a14b5c66d31c6eac65962bf54ea" exitCode=0
Oct 30 12:57:42.881129 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.881105    5781 generic.go:356] "Generic (PLEG): container finished" podID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f" exitCode=0
Oct 30 12:57:42.892455 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.892399    5781 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="0e3012534e87b6e45f60966061605ac0050f5cf6baefaeccfe18bedfcccb05c3" exitCode=0
Oct 30 12:57:42.892455 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.892423    5781 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="ae248779d1c2d6205f2830472f027dd9a5de172c1e3cfcf8775607d9ec4dcded" exitCode=0
Oct 30 12:57:42.892455 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.892433    5781 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="6b242cff777197d1224eeb50c91a017629c67077b1e1415d3ffecac07a13fb74" exitCode=0
Oct 30 12:57:42.892455 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.892440    5781 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="5ccd197fa405996a479011b4194a7aa634d29b979603cbe064b12bc13fcbf9ee" exitCode=0
Oct 30 12:57:42.892455 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.892445    5781 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="8c97bddc001d013ac39da9299c0dacbedcb82d2440c3e0318ab9ade053328a30" exitCode=0
Oct 30 12:57:42.892455 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.892451    5781 generic.go:356] "Generic (PLEG): container finished" podID="f4471adf-01e8-47ae-8056-c670541a735c" containerID="8112e4ae72f183aa07e50045111c4f206b3cb740823e9c871a5c9a80780f3056" exitCode=0
Oct 30 12:57:42.926544 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.926524    5781 manager.go:324] Recovery completed
Oct 30 12:57:42.933497 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:57:42.933472    5781 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 12:57:42.950121 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950091    5781 cpu_manager.go:226] "Starting" policy="none"
Oct 30 12:57:42.950121 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950107    5781 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 12:57:42.950121 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950127    5781 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 12:57:42.950320 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950261    5781 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 12:57:42.950320 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950273    5781 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 12:57:42.950401 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950347    5781 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 12:57:42.950401 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950354    5781 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 12:57:42.950401 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950360    5781 policy_none.go:50] "Start"
Oct 30 12:57:42.950401 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950374    5781 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 12:57:42.950401 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.950385    5781 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 12:57:42.951566 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.951550    5781 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 12:57:42.951609 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.951569    5781 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 12:57:42.951609 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.951577    5781 policy_none.go:44] "Start"
Oct 30 12:57:42.958419 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.957869    5781 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 12:57:42.963281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.963263    5781 manager.go:341] "Starting Device Plugin manager"
Oct 30 12:57:42.963409 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:57:42.963314    5781 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 12:57:42.963409 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.963327    5781 server.go:87] "Starting device plugin registration server"
Oct 30 12:57:42.963552 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.963540    5781 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 12:57:42.963611 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.963554    5781 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 12:57:42.963658 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.963644    5781 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 12:57:42.963787 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.963755    5781 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 12:57:42.963841 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.963795    5781 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 12:57:42.963841 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.963802    5781 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 12:57:42.963934 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.963843    5781 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T12:57:42.963784269Z","UUID":"4474b036-bc46-4b3b-b12d-43f1d0cd8d9b","Handler":null,"Name":"","Endpoint":""}
Oct 30 12:57:42.965253 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.965233    5781 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 12:57:42.965389 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:42.965273    5781 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 12:57:42.966195 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:57:42.966170    5781 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 12:57:43.064155 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.064115    5781 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:57:43.066087 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.066064    5781 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:57:43.066208 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.066105    5781 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:57:43.066208 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.066120    5781 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-180.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:57:43.066208 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.066180    5781 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.116821 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.116795    5781 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.116924 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.116868    5781 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.133942 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.133899    5781 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"]
Oct 30 12:57:43.134530 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.134513    5781 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.134612 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.134534    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal" event={"ID":"9b3f18c352b4e52d3d8a287983d93c2b","Type":"ContainerStarted","Data":"b538f5c85104384a835e754e6fa585c049abcf5ef8076f553365c8704390d433"}
Oct 30 12:57:43.134612 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.134568    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal" event={"ID":"9b3f18c352b4e52d3d8a287983d93c2b","Type":"ContainerDied","Data":"543935082c87ae7e02833bd8ed70e93deac60a14b5c66d31c6eac65962bf54ea"}
Oct 30 12:57:43.134612 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.134582    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal" event={"ID":"9b3f18c352b4e52d3d8a287983d93c2b","Type":"ContainerStarted","Data":"a6d797fb42450ffe205b46017cda83bb21b1a4e8e3ea7d054661bcca72bb445b"}
Oct 30 12:57:43.134612 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.134603    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal" event={"ID":"dbd004dde5b9ed65a173b78115c079e0","Type":"ContainerStarted","Data":"205ac379ad661cb070f4d3c8c022adf712cde46fa98d5a1ecd97ddcb92117ccc"}
Oct 30 12:57:43.134793 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.134614    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal" event={"ID":"dbd004dde5b9ed65a173b78115c079e0","Type":"ContainerStarted","Data":"914daae99f98a21338ff4a5f32aba5ef1c8376b8411f53722835adbe3af5a601"}
Oct 30 12:57:43.134793 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.134708    5781 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.169366 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.169342    5781 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:57:43.169473 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:57:43.169392    5781 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-180.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.172076 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.172061    5781 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:57:43.172122 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:57:43.172111    5781 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.246401 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.246371    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbd004dde5b9ed65a173b78115c079e0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-180.ec2.internal\" (UID: \"dbd004dde5b9ed65a173b78115c079e0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.246537 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.246406    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.246537 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.246426    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.346963 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.346929    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbd004dde5b9ed65a173b78115c079e0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-180.ec2.internal\" (UID: \"dbd004dde5b9ed65a173b78115c079e0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.346963 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.346963    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.347137 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.346981    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.347137 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.347031    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.347137 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.347035    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dbd004dde5b9ed65a173b78115c079e0-config\") pod \"kube-apiserver-proxy-ip-10-0-130-180.ec2.internal\" (UID: \"dbd004dde5b9ed65a173b78115c079e0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.347137 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.347110    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b3f18c352b4e52d3d8a287983d93c2b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" (UID: \"9b3f18c352b4e52d3d8a287983d93c2b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.772994 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.772960    5781 apiserver.go:52] "Watching apiserver"
Oct 30 12:57:43.823452 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.823418    5781 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 12:57:43.824134 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.824103    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz","openshift-monitoring/prometheus-operator-9cd89b74c-bq5df","openshift-monitoring/node-exporter-bkk45","openshift-image-registry/node-ca-t7zb4","openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k","openshift-cluster-node-tuning-operator/tuned-5gn6d","openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh","openshift-multus/multus-additional-cni-plugins-97p84","openshift-monitoring/prometheus-k8s-0","openshift-multus/multus-k6rf5","kube-system/global-pull-secret-syncer-gkrqb","kube-system/konnectivity-agent-c56xj","openshift-monitoring/thanos-querier-dcc9fd45-6nh5c","kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal","openshift-dns/dns-default-fqj4w","openshift-dns/node-resolver-rlghf","openshift-ingress-canary/ingress-canary-gphtb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-nzs4p","openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2","openshift-multus/network-metrics-daemon-xtz6h","openshift-network-diagnostics/network-check-target-xc6hk","openshift-insights/insights-runtime-extractor-qlp5s","openshift-network-operator/iptables-alerter-sdmv9","openshift-monitoring/alertmanager-main-0"]
Oct 30 12:57:43.841649 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.841611    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 12:57:43.841649 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.841635    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xt92t\""
Oct 30 12:57:43.841995 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.841980    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v4tv9\""
Oct 30 12:57:43.842114 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842088    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-wsrzk\""
Oct 30 12:57:43.842193 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842136    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7gdbz\""
Oct 30 12:57:43.842270 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842237    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.842270 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842239    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 12:57:43.842270 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842250    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.842482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842331    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.842482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842401    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.842482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842411    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.842482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842417    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 12:57:43.842482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842416    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 12:57:43.842482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842430    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 12:57:43.842482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842418    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.842482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842444    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 12:57:43.842482 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842404    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 12:57:43.842764 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.842629    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 12:57:43.848497 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.848479    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-vg7bt\""
Oct 30 12:57:43.848610 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.848539    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dz42j\""
Oct 30 12:57:43.848610 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.848582    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-cxt4s\""
Oct 30 12:57:43.848757 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.848741    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.849186 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.849156    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.849186 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.849179    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 12:57:43.849308 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.849254    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.850589 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.850574    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.850739 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.850721    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.851179 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.851162    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.851450 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.851421    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.851550 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.851449    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.851550 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.851486    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.851550 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.851521    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 12:57:43.851703 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.851576    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kmvh7\""
Oct 30 12:57:43.853477 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.853458    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.853564 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.853482    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-94phb\""
Oct 30 12:57:43.853564 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.853493    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tzbll\""
Oct 30 12:57:43.853564 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.853513    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-sp79l\""
Oct 30 12:57:43.853564 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.853512    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-25fk6\""
Oct 30 12:57:43.856137 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.856122    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 12:57:43.856351 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.856337    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.857091 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857079    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.857145 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857125    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 12:57:43.857214 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857197    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.857329 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857244    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 12:57:43.857404 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857373    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 12:57:43.857404 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857402    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 12:57:43.857539 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857407    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.857539 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857420    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 12:57:43.857539 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857426    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 12:57:43.857539 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857464    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 12:57:43.857539 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857474    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 12:57:43.857539 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.857518    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 12:57:43.862815 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.862796    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 12:57:43.862933 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.862859    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 12:57:43.862933 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.862901    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 12:57:43.862933 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.862927    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 12:57:43.863139 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.862955    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 12:57:43.863139 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.862987    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 12:57:43.863139 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.862987    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-sqtw6\""
Oct 30 12:57:43.863139 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.863043    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5bbfv\""
Oct 30 12:57:43.863761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.863719    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 12:57:43.863966 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.863943    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2qk6c\""
Oct 30 12:57:43.863966 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.863953    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 12:57:43.864117 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.864006    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 12:57:43.864117 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.864050    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-skpfr\""
Oct 30 12:57:43.864117 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.864080    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 12:57:43.864117 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.864097    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 12:57:43.864311 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.864129    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 12:57:43.864311 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.864141    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 12:57:43.864311 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.864235    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 12:57:43.865029 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.865013    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 12:57:43.865110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.865057    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 12:57:43.867196 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867177    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 12:57:43.867434 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867412    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 12:57:43.867541 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867425    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-sdfnl\""
Oct 30 12:57:43.867541 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867422    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 12:57:43.867541 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867482    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 12:57:43.867541 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867501    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 12:57:43.867742 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867613    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 12:57:43.867742 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867684    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 12:57:43.867834 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867761    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 12:57:43.867834 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867795    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 12:57:43.867834 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867795    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 12:57:43.867834 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867810    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-h2dkw\""
Oct 30 12:57:43.868011 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.867934    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-b82bm\""
Oct 30 12:57:43.868332 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.868318    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6gqfl\""
Oct 30 12:57:43.868439 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.868330    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cacau0ihglokf\""
Oct 30 12:57:43.868439 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.868385    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 12:57:43.868439 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.868425    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 12:57:43.868532 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.868466    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 12:57:43.868563 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.868532    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 12:57:43.869005 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.868983    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 12:57:43.869005 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.868999    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 12:57:43.869134 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.869014    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 12:57:43.869239 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.869212    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 12:57:43.869479 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.869462    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-59dum1jo40hp4\""
Oct 30 12:57:43.869635 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.869594    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 12:57:43.869690 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.869647    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 12:57:43.869760 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.869625    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 12:57:43.869872 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.869856    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 12:57:43.870150 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.870128    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 12:57:43.870368 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.870353    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 12:57:43.870466 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.870439    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2665n\""
Oct 30 12:57:43.870699 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.870682    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 12:57:43.870785 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.870701    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 12:57:43.870911 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.870890    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 12:57:43.871248 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.871158    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 12:57:43.871341 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.871270    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5ikgvvfbsde4u\""
Oct 30 12:57:43.871443 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.871425    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 12:57:43.871657 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.871642    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-82464\""
Oct 30 12:57:43.871709 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.871650    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-djsc9\""
Oct 30 12:57:43.871709 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.871674    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pd6vw\""
Oct 30 12:57:43.872565 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.872544    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 12:57:43.901231 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.901209    5781 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.901347 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.901312    5781 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.907196 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.907177    5781 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 12:57:43.925142 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.925120    5781 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:57:43.925277 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:57:43.925177    5781 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-180.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.925935 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.925914    5781 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:57:43.926018 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:57:43.925958    5781 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-180.ec2.internal"
Oct 30 12:57:43.949855 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.949831    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.949978 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.949864    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.949978 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.949908    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-registration-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:43.950148 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950127    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4c5v\" (UniqueName: \"kubernetes.io/projected/1062c6b4-6828-4e92-92a5-c78eccb1fe26-kube-api-access-c4c5v\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:43.950208 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950164    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2926b50e-e07d-445d-bbec-eab0005a59bf-host\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:57:43.950208 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950179    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.950314 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950215    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-tmp\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.950373 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950314    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.950373 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950340    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-systemd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.950373 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950368    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-tmp\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.950517 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950423    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-log-socket\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.950517 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950442    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-system-cni-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.950517 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950482    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.950724 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950514    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqs7p\" (UniqueName: \"kubernetes.io/projected/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kube-api-access-jqs7p\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:43.950724 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950541    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39289142-2079-4b45-ba09-8e06aed12276-konnectivity-ca\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:57:43.950724 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950573    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-run\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.950724 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950596    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:43.950724 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950623    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.950724 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950692    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950736    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950769    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950797    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74s7t\" (UniqueName: \"kubernetes.io/projected/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-api-access-74s7t\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950798    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/39289142-2079-4b45-ba09-8e06aed12276-konnectivity-ca\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950812    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950848    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-netns\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950848    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950889    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950916    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-etc-selinux\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:43.951002 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.950969    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951029    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87s78\" (UniqueName: \"kubernetes.io/projected/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-kube-api-access-87s78\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951035    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951061    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951064    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951098    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-cni-binary-copy\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951265    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951270    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1062c6b4-6828-4e92-92a5-c78eccb1fe26-metrics-certs\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951279    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951320    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-cni-binary-copy\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951340    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951366    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951382    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6214326d-31ce-4577-bd4b-7997c910a058-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951402    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4bf5\" (UniqueName: \"kubernetes.io/projected/6214326d-31ce-4577-bd4b-7997c910a058-kube-api-access-w4bf5\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:43.951523 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951497    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951550    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951570    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6214326d-31ce-4577-bd4b-7997c910a058-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951579    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951608    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951612    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951707    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951716    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951754    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853421d2-ef94-4cbd-b251-1f4d82bfb45e-tmp-dir\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951782    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-sys\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951793    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951809    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951861    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-bin\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951862    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853421d2-ef94-4cbd-b251-1f4d82bfb45e-tmp-dir\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951885    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951887    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-cnibin\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951873    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-original-pull-secret\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951888    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.951990    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952027    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpjcn\" (UniqueName: \"kubernetes.io/projected/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-kube-api-access-vpjcn\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:43.952110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952049    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952148    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952172    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952234    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-root\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952338    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952468    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952470    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br287\" (UniqueName: \"kubernetes.io/projected/1a451455-21be-4f3a-a56d-8c4026bcf654-kube-api-access-br287\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952520    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-netns\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952548    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952549    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-accelerators-collector-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952584    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-var-lib-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952643    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952699    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-wtmp\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952728    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952806    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952844    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-etc-kubernetes\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952854    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952862    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/162db930-0919-4366-ac55-b8e647cb714b-ovn-node-metrics-cert\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952879    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/793c2518-45fa-46e0-9a23-766ae40d922b-tmp-dir\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:43.952979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952973    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-cert\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.952991    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953003    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-sys-fs\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953035    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/793c2518-45fa-46e0-9a23-766ae40d922b-tmp-dir\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953053    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lh9t\" (UniqueName: \"kubernetes.io/projected/2926b50e-e07d-445d-bbec-eab0005a59bf-kube-api-access-8lh9t\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953079    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-crio-socket\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953104    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953120    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-client-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953174    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953212    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/162db930-0919-4366-ac55-b8e647cb714b-ovn-node-metrics-cert\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953210    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953271    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f8c0d1-d514-488c-9401-b745e93ad9ac-metrics-client-ca\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953319    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-grpc-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953345    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-host\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953372    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953399    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853421d2-ef94-4cbd-b251-1f4d82bfb45e-hosts-file\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953423    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953425    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953445    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-socket-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953451    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-cert\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953471    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8cc6455-76b2-40a2-aa43-5b516d85d543-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953516    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-client-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953521    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953647    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953671    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-grpc-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953677    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953681    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b8cc6455-76b2-40a2-aa43-5b516d85d543-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953682    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953695    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/99f8c0d1-d514-488c-9401-b745e93ad9ac-metrics-client-ca\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953736    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-system-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953763    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-dbus\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953764    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953807    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953834    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.953836 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953857    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.953867    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954000    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954049    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954208    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954247    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954276    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-device-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954328    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-cnibin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954354    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954382    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954414    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/793c2518-45fa-46e0-9a23-766ae40d922b-metrics-tls\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954438    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954462    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb64l\" (UniqueName: \"kubernetes.io/projected/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-api-access-jb64l\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954487    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954511    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954539    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954565    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954586    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-env-overrides\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954618    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954645    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-tls\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954669    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954694    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-systemd-units\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954719    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hskxz\" (UniqueName: \"kubernetes.io/projected/99f8c0d1-d514-488c-9401-b745e93ad9ac-kube-api-access-hskxz\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954742    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-hostroot\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954768    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-multus-daemon-config\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954797    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7nwx\" (UniqueName: \"kubernetes.io/projected/0adce241-e2df-40ec-8976-341e00ebc074-kube-api-access-v7nwx\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954821    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysconfig\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954844    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-kubernetes\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.954877 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954867    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954892    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-config\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954915    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954941    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954969    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-metrics-server-audit-profiles\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.954994    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955034    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-var-lib-kubelet\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955063    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955089    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-binary-copy\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955115    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955139    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-metrics-client-ca\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955166    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955191    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99f8c0d1-d514-488c-9401-b745e93ad9ac-tmp\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955215    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-multus\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955244    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nhbx\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955282    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955326    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-ovn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955351    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-node-log\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955490    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955572    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955577    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955715    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955718    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0adce241-e2df-40ec-8976-341e00ebc074-multus-daemon-config\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955803    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-binary-copy\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955862    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955872    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a451455-21be-4f3a-a56d-8c4026bcf654-metrics-client-ca\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955875    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/99f8c0d1-d514-488c-9401-b745e93ad9ac-tmp\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.955947 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.955901    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956034    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-env-overrides\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956081    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-tls\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956097    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956146    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956152    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956241    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956334    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956437    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956481    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956514    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-tls\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956526    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-config\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956542    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39289142-2079-4b45-ba09-8e06aed12276-agent-certs\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956557    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/793c2518-45fa-46e0-9a23-766ae40d922b-metrics-tls\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956666    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956673    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956687    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956706    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2926b50e-e07d-445d-bbec-eab0005a59bf-serviceca\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956773    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/39289142-2079-4b45-ba09-8e06aed12276-agent-certs\") pod \"konnectivity-agent-c56xj\" (UID: \"39289142-2079-4b45-ba09-8e06aed12276\") " pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956833    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-systemd\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956861    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-lib-modules\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956884    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-tls\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956904    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-data-volume\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956931    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-kubelet\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956936    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/88f01f9c-02af-4e17-9a80-7f769221c703-metrics-server-audit-profiles\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.956958    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/793c2518-45fa-46e0-9a23-766ae40d922b-config-volume\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:43.957042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957041    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-data-volume\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957084    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xhvx\" (UniqueName: \"kubernetes.io/projected/793c2518-45fa-46e0-9a23-766ae40d922b-kube-api-access-7xhvx\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957116    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqzpn\" (UniqueName: \"kubernetes.io/projected/162db930-0919-4366-ac55-b8e647cb714b-kube-api-access-cqzpn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957145    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-iptables-alerter-script\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957176    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957229    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8jdj\" (UniqueName: \"kubernetes.io/projected/88f01f9c-02af-4e17-9a80-7f769221c703-kube-api-access-t8jdj\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957336    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l97l\" (UniqueName: \"kubernetes.io/projected/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-kube-api-access-5l97l\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957366    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957395    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957412    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2926b50e-e07d-445d-bbec-eab0005a59bf-serviceca\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957437    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957454    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957469    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-conf-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957509    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgd46\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957541    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-socket-dir-parent\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.957858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957822    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/99f8c0d1-d514-488c-9401-b745e93ad9ac-secret-thanos-querier-tls\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957862    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpbk9\" (UniqueName: \"kubernetes.io/projected/b8cc6455-76b2-40a2-aa43-5b516d85d543-kube-api-access-wpbk9\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957892    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-netd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957921    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v5gg\" (UniqueName: \"kubernetes.io/projected/853421d2-ef94-4cbd-b251-1f4d82bfb45e-kube-api-access-2v5gg\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957949    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-host-slash\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.957974    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-etc-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958001    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958029    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958054    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-slash\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958083    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958111    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-kubelet-config\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958139    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/88f01f9c-02af-4e17-9a80-7f769221c703-audit-log\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.958276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958165    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-sys\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958339    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-bin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958516    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958553    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/88f01f9c-02af-4e17-9a80-7f769221c703-audit-log\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958566    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958587    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958592    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-conf\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958662    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f4471adf-01e8-47ae-8056-c670541a735c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958733    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-tuned\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958770    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kh5j\" (UniqueName: \"kubernetes.io/projected/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-kube-api-access-8kh5j\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958846    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgt5q\" (UniqueName: \"kubernetes.io/projected/f4471adf-01e8-47ae-8056-c670541a735c-kube-api-access-bgt5q\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958887    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-textfile\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958900    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-iptables-alerter-script\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958970    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958976    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/793c2518-45fa-46e0-9a23-766ae40d922b-config-volume\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:43.958987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.958989    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-tuned\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959009    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-k8s-cni-cncf-io\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959014    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959048    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-textfile\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959086    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-kubelet\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959207    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-multus-certs\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959220    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959254    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959265    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959320    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-script-lib\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959400    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-client-certs\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959462    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-modprobe-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959574    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/162db930-0919-4366-ac55-b8e647cb714b-ovnkube-script-lib\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.959731 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959727    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/88f01f9c-02af-4e17-9a80-7f769221c703-secret-metrics-server-client-certs\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.960318 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.959783    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6214326d-31ce-4577-bd4b-7997c910a058-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:43.971886 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.971865    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4c5v\" (UniqueName: \"kubernetes.io/projected/1062c6b4-6828-4e92-92a5-c78eccb1fe26-kube-api-access-c4c5v\") pod \"network-metrics-daemon-xtz6h\" (UID: \"1062c6b4-6828-4e92-92a5-c78eccb1fe26\") " pod="openshift-multus/network-metrics-daemon-xtz6h"
Oct 30 12:57:43.972007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.971966    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4bf5\" (UniqueName: \"kubernetes.io/projected/6214326d-31ce-4577-bd4b-7997c910a058-kube-api-access-w4bf5\") pod \"openshift-state-metrics-7bd7489957-jmsqh\" (UID: \"6214326d-31ce-4577-bd4b-7997c910a058\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-jmsqh"
Oct 30 12:57:43.972073 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.972043    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87s78\" (UniqueName: \"kubernetes.io/projected/b5c3d5df-2428-490f-925a-f67ef2e8c0b6-kube-api-access-87s78\") pod \"ingress-canary-gphtb\" (UID: \"b5c3d5df-2428-490f-925a-f67ef2e8c0b6\") " pod="openshift-ingress-canary/ingress-canary-gphtb"
Oct 30 12:57:43.972446 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.972427    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqs7p\" (UniqueName: \"kubernetes.io/projected/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kube-api-access-jqs7p\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:43.972539 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.972506    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74s7t\" (UniqueName: \"kubernetes.io/projected/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-kube-api-access-74s7t\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:43.972792 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.972774    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpjcn\" (UniqueName: \"kubernetes.io/projected/03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa-kube-api-access-vpjcn\") pod \"prometheus-operator-9cd89b74c-bq5df\" (UID: \"03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-bq5df"
Oct 30 12:57:43.973531 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.973515    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-br287\" (UniqueName: \"kubernetes.io/projected/1a451455-21be-4f3a-a56d-8c4026bcf654-kube-api-access-br287\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:43.976386 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.976368    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lh9t\" (UniqueName: \"kubernetes.io/projected/2926b50e-e07d-445d-bbec-eab0005a59bf-kube-api-access-8lh9t\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:57:43.981425 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.981387    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xhvx\" (UniqueName: \"kubernetes.io/projected/793c2518-45fa-46e0-9a23-766ae40d922b-kube-api-access-7xhvx\") pod \"dns-default-fqj4w\" (UID: \"793c2518-45fa-46e0-9a23-766ae40d922b\") " pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:43.982110 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.982092    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l97l\" (UniqueName: \"kubernetes.io/projected/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-kube-api-access-5l97l\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:57:43.982276 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.982254    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kh5j\" (UniqueName: \"kubernetes.io/projected/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-kube-api-access-8kh5j\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:43.982445 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.982423    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7nwx\" (UniqueName: \"kubernetes.io/projected/0adce241-e2df-40ec-8976-341e00ebc074-kube-api-access-v7nwx\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:43.982495 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.982455    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nhbx\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx\") pod \"alertmanager-main-0\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:57:43.982533 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.982492    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb64l\" (UniqueName: \"kubernetes.io/projected/c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4-kube-api-access-jb64l\") pod \"kube-state-metrics-5dc8d7d89b-7l2k2\" (UID: \"c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-7l2k2"
Oct 30 12:57:43.982571 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.982550    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srms2\" (UniqueName: \"kubernetes.io/projected/4faa20e7-dd05-4416-9b9e-ed52b91d388d-kube-api-access-srms2\") pod \"network-check-target-xc6hk\" (UID: \"4faa20e7-dd05-4416-9b9e-ed52b91d388d\") " pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:43.982808 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.982791    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgt5q\" (UniqueName: \"kubernetes.io/projected/f4471adf-01e8-47ae-8056-c670541a735c-kube-api-access-bgt5q\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:43.983093 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.983073    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hskxz\" (UniqueName: \"kubernetes.io/projected/99f8c0d1-d514-488c-9401-b745e93ad9ac-kube-api-access-hskxz\") pod \"thanos-querier-dcc9fd45-6nh5c\" (UID: \"99f8c0d1-d514-488c-9401-b745e93ad9ac\") " pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:43.983272 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.983258    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8jdj\" (UniqueName: \"kubernetes.io/projected/88f01f9c-02af-4e17-9a80-7f769221c703-kube-api-access-t8jdj\") pod \"metrics-server-6dd9dd5bc9-k4v6k\" (UID: \"88f01f9c-02af-4e17-9a80-7f769221c703\") " pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:43.983400 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.983385    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqzpn\" (UniqueName: \"kubernetes.io/projected/162db930-0919-4366-ac55-b8e647cb714b-kube-api-access-cqzpn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:43.986813 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.986797    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgd46\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46\") pod \"prometheus-k8s-0\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:43.987260 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.987244    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v5gg\" (UniqueName: \"kubernetes.io/projected/853421d2-ef94-4cbd-b251-1f4d82bfb45e-kube-api-access-2v5gg\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:57:43.990639 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:43.990623    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpbk9\" (UniqueName: \"kubernetes.io/projected/b8cc6455-76b2-40a2-aa43-5b516d85d543-kube-api-access-wpbk9\") pod \"monitoring-plugin-77f4f76594-cdcx5\" (UID: \"b8cc6455-76b2-40a2-aa43-5b516d85d543\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:44.060386 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060307    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-conf-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060386 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060341    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-socket-dir-parent\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060386 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060358    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-netd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.060386 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060372    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-host-slash\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:57:44.060386 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060386    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-etc-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060416    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-conf-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060436    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-socket-dir-parent\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060436    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-netd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060439    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15d363e1-d64d-4ab4-a134-08ef9b8dd9f9-host-slash\") pod \"iptables-alerter-sdmv9\" (UID: \"15d363e1-d64d-4ab4-a134-08ef9b8dd9f9\") " pod="openshift-network-operator/iptables-alerter-sdmv9"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060423    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-etc-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060494    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-slash\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060518    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-kubelet-config\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060534    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-sys\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060551    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-bin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060566    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-conf\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060589    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-kubelet-config\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060592    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-sys\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060590    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-k8s-cni-cncf-io\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060615    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-bin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060621    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-slash\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060627    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-kubelet\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060655    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-multus-certs\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060659    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-kubelet\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060679    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-multus-certs\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060681    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-k8s-cni-cncf-io\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060678    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-modprobe-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.060717 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060709    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-registration-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060728    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2926b50e-e07d-445d-bbec-eab0005a59bf-host\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060718    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-conf\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060764    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-systemd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060771    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2926b50e-e07d-445d-bbec-eab0005a59bf-host\") pod \"node-ca-t7zb4\" (UID: \"2926b50e-e07d-445d-bbec-eab0005a59bf\") " pod="openshift-image-registry/node-ca-t7zb4"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060773    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-modprobe-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060787    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-registration-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060796    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-systemd\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060790    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-log-socket\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060816    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-log-socket\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060826    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-system-cni-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060842    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-run\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060859    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060876    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-system-cni-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060886    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-netns\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060901    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-run\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060918    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-multus-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060931    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-etc-selinux\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060905    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-etc-selinux\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060946    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-netns\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.060987    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-sys\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061012    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-bin\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061017    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-sys\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061035    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-cnibin\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061042    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-cni-bin\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061061    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-root\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061083    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-cnibin\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061090    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-netns\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061107    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-root\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061114    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-var-lib-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061139    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-run-netns\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061146    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-wtmp\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061171    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-etc-kubernetes\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061180    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1a451455-21be-4f3a-a56d-8c4026bcf654-node-exporter-wtmp\") pod \"node-exporter-bkk45\" (UID: \"1a451455-21be-4f3a-a56d-8c4026bcf654\") " pod="openshift-monitoring/node-exporter-bkk45"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061199    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-sys-fs\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061147    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-var-lib-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061223    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-etc-kubernetes\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061225    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-crio-socket\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061255    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-host\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061279    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853421d2-ef94-4cbd-b251-1f4d82bfb45e-hosts-file\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061319    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a9daaf5-4ea6-472e-9972-68cfeee5ae16-crio-socket\") pod \"insights-runtime-extractor-qlp5s\" (UID: \"4a9daaf5-4ea6-472e-9972-68cfeee5ae16\") " pod="openshift-insights/insights-runtime-extractor-qlp5s"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061319    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061335    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-host\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061345    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-socket-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061365    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853421d2-ef94-4cbd-b251-1f4d82bfb45e-hosts-file\") pod \"node-resolver-rlghf\" (UID: \"853421d2-ef94-4cbd-b251-1f4d82bfb45e\") " pod="openshift-dns/node-resolver-rlghf"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061367    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-system-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061392    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-system-cni-dir\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061400    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-dbus\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061427    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-device-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061456    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-cnibin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061481    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061486    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-device-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061456    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061496    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c69d9666-d791-48fe-aad3-7a91f2ab2d2c-dbus\") pod \"global-pull-secret-syncer-gkrqb\" (UID: \"c69d9666-d791-48fe-aad3-7a91f2ab2d2c\") " pod="kube-system/global-pull-secret-syncer-gkrqb"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061458    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-socket-dir\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061520    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-cnibin\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061544    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-run-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061572    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-systemd-units\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061590    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-hostroot\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061593    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ed7b27fb-3f62-473c-89d0-ea4ced2bbe56-sys-fs\") pod \"aws-ebs-csi-driver-node-stpzz\" (UID: \"ed7b27fb-3f62-473c-89d0-ea4ced2bbe56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-stpzz"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061596    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-systemd-units\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061606    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysconfig\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061628    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-kubernetes\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061632    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysconfig\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061656    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-hostroot\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061662    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-kubernetes\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061656    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061685    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061702    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-var-lib-kubelet\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061724    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061754    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-var-lib-kubelet\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.061698 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061753    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f4471adf-01e8-47ae-8056-c670541a735c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97p84\" (UID: \"f4471adf-01e8-47ae-8056-c670541a735c\") " pod="openshift-multus/multus-additional-cni-plugins-97p84"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061754    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-multus\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061765    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-sysctl-d\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061777    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061788    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061793    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0adce241-e2df-40ec-8976-341e00ebc074-host-var-lib-cni-multus\") pod \"multus-k6rf5\" (UID: \"0adce241-e2df-40ec-8976-341e00ebc074\") " pod="openshift-multus/multus-k6rf5"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061807    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-openvswitch\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061813    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-ovn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061833    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-node-log\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061837    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-run-ovn\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061852    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-systemd\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061866    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-lib-modules\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061885    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-etc-systemd\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061901    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-kubelet\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061906    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-node-log\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061949    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/162db930-0919-4366-ac55-b8e647cb714b-host-kubelet\") pod \"ovnkube-node-nzs4p\" (UID: \"162db930-0919-4366-ac55-b8e647cb714b\") " pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:44.063816 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.061951    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d-lib-modules\") pod \"tuned-5gn6d\" (UID: \"d8f1ca79-0b5a-4a41-a574-e2bdb4a9a94d\") " pod="openshift-cluster-node-tuning-operator/tuned-5gn6d"
Oct 30 12:57:44.140820 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.140801    5781 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:57:44.522125 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.522090    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:44.529460 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.529436    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-dcc9fd45-6nh5c"
Oct 30 12:57:44.762866 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.762824    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:44.767023 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:44.766995    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-cdcx5"
Oct 30 12:57:45.298529 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:45.298491    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:57:45.853354 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:45.853320    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fqj4w"
Oct 30 12:57:45.993054 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:45.993022    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:46.148620 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:46.148540    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:46.975599 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:46.975572    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c56xj"
Oct 30 12:57:47.327402 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:47.327383    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:47.723349 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:47.723327    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:47.935215 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:47.935173    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerStarted","Data":"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"}
Oct 30 12:57:47.935409 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:47.935320    5781 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 12:57:47.935468 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:47.935319    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerStarted","Data":"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"}
Oct 30 12:57:47.935543 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:47.935522    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerStarted","Data":"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"}
Oct 30 12:57:47.935617 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:47.935576    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerStarted","Data":"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"}
Oct 30 12:57:47.935671 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:47.935625    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerStarted","Data":"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"}
Oct 30 12:57:47.935725 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:47.935667    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerStarted","Data":"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"}
Oct 30 12:57:48.277091 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:48.277021    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nzs4p"
Oct 30 12:57:49.293646 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:49.293614    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:49.436617 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:49.436587    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:51.285628 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:51.285592    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:57:51.896297 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:51.896258    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:57:52.126864 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:52.126835    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:52.278042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:52.278018    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:52.863061 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:52.863025    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:53.023466 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:53.023435    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:53.191924 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:53.191883    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:53.353786 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:53.353737    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:54.032946 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:54.032918    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:54.230570 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:54.230522    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:56.472607 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:56.472568    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:56.629833 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:56.629783    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:56.955649 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:56.955596    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:57:57.095172 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:57:57.095081    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:02.000263 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:02.000221    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 12:58:03.785075 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:03.785041    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:58:11.290974 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:11.290941    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:58:11.294775 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:11.294752    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6dd9dd5bc9-k4v6k"
Oct 30 12:58:13.705624 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:13.705592    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:13.724362 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:13.724332    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:14.058691 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:14.058622    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:15.301210 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:15.301181    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xc6hk"
Oct 30 12:58:17.621747 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:17.621707    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:17.770790 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:17.770759    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:24.212861 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:24.212825    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:24.487579 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:24.487501    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:25.507841 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:25.507797    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 12:58:25.508374 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:25.508185    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="alertmanager" containerID="cri-o://cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb" gracePeriod=120
Oct 30 12:58:25.508374 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:25.508234    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy-metric" containerID="cri-o://8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b" gracePeriod=120
Oct 30 12:58:25.508374 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:25.508302    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy-web" containerID="cri-o://4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3" gracePeriod=120
Oct 30 12:58:25.508374 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:25.508305    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="config-reloader" containerID="cri-o://308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a" gracePeriod=120
Oct 30 12:58:25.508374 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:25.508319    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy" containerID="cri-o://b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069" gracePeriod=120
Oct 30 12:58:25.508374 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:25.508335    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="prom-label-proxy" containerID="cri-o://ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240" gracePeriod=120
Oct 30 12:58:26.089754 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089718    5781 generic.go:356] "Generic (PLEG): container finished" podID="0855ce73-dcfd-4398-b658-301a73ea990d" containerID="ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240" exitCode=0
Oct 30 12:58:26.089754 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089744    5781 generic.go:356] "Generic (PLEG): container finished" podID="0855ce73-dcfd-4398-b658-301a73ea990d" containerID="8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b" exitCode=0
Oct 30 12:58:26.089754 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089750    5781 generic.go:356] "Generic (PLEG): container finished" podID="0855ce73-dcfd-4398-b658-301a73ea990d" containerID="b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069" exitCode=0
Oct 30 12:58:26.089754 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089759    5781 generic.go:356] "Generic (PLEG): container finished" podID="0855ce73-dcfd-4398-b658-301a73ea990d" containerID="308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a" exitCode=0
Oct 30 12:58:26.089754 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089765    5781 generic.go:356] "Generic (PLEG): container finished" podID="0855ce73-dcfd-4398-b658-301a73ea990d" containerID="cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb" exitCode=0
Oct 30 12:58:26.090054 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089790    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerDied","Data":"ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240"}
Oct 30 12:58:26.090054 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089854    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerDied","Data":"8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b"}
Oct 30 12:58:26.090054 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089879    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerDied","Data":"b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069"}
Oct 30 12:58:26.090054 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089904    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerDied","Data":"308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a"}
Oct 30 12:58:26.090054 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.089925    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerDied","Data":"cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb"}
Oct 30 12:58:26.477513 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.477480    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:26.628230 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.628132    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:26.762061 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.762032    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:26.834273 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834241    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834273 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834278    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834528 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834313    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834528 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834457    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834636 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834524    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834636 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834556    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834636 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834594    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834784 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834634    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834784 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834664    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834784 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834732    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834784 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834772    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834974 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834835    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.834974 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.834866    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web\") pod \"0855ce73-dcfd-4398-b658-301a73ea990d\" (UID: \"0855ce73-dcfd-4398-b658-301a73ea990d\") "
Oct 30 12:58:26.835273 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.835240    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:58:26.835992 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.835677    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:26.838811 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.838673    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:26.838811 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.838677    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:26.838811 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.838708    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:58:26.839106 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.839079    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:58:26.841493 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.841398    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "kube-api-access-4nhbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:58:26.841493 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.841439    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:26.845147 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.845094    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:26.845428 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.845385    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:26.846854 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.846828    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:26.847127 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.847099    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:26.854875 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.854842    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config" pod "0855ce73-dcfd-4398-b658-301a73ea990d" (UID: "0855ce73-dcfd-4398-b658-301a73ea990d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:26.935559 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935526    5781 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935559 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935555    5781 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-cluster-tls-config\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935568    5781 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935577    5781 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935588    5781 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-tls-assets\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935596    5781 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935604    5781 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-alertmanager-main-db\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935613    5781 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0855ce73-dcfd-4398-b658-301a73ea990d-metrics-client-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935621    5781 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-web-config\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935629    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nhbx\" (UniqueName: \"kubernetes.io/projected/0855ce73-dcfd-4398-b658-301a73ea990d-kube-api-access-4nhbx\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935638    5781 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-main-tls\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935646    5781 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0855ce73-dcfd-4398-b658-301a73ea990d-config-out\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:26.935761 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:26.935654    5781 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0855ce73-dcfd-4398-b658-301a73ea990d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:27.096701 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.096614    5781 generic.go:356] "Generic (PLEG): container finished" podID="0855ce73-dcfd-4398-b658-301a73ea990d" containerID="4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3" exitCode=0
Oct 30 12:58:27.096701 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.096678    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerDied","Data":"4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3"}
Oct 30 12:58:27.096891 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.096707    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.096891 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.096761    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"0855ce73-dcfd-4398-b658-301a73ea990d","Type":"ContainerDied","Data":"cd49119309dcb11aab8d8f869b8f8dc0fae222844bc784f4d5ae31d20b599ffb"}
Oct 30 12:58:27.096891 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.096794    5781 scope.go:122] "RemoveContainer" containerID="ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240"
Oct 30 12:58:27.105437 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.105421    5781 scope.go:122] "RemoveContainer" containerID="8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b"
Oct 30 12:58:27.112168 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.112147    5781 scope.go:122] "RemoveContainer" containerID="b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069"
Oct 30 12:58:27.118950 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.118933    5781 scope.go:122] "RemoveContainer" containerID="4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3"
Oct 30 12:58:27.125679 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.125647    5781 scope.go:122] "RemoveContainer" containerID="308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a"
Oct 30 12:58:27.132543 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.132528    5781 scope.go:122] "RemoveContainer" containerID="cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb"
Oct 30 12:58:27.139154 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.139131    5781 scope.go:122] "RemoveContainer" containerID="7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f"
Oct 30 12:58:27.145756 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.145730    5781 scope.go:122] "RemoveContainer" containerID="ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240"
Oct 30 12:58:27.146016 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:27.145996    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240\": container with ID starting with ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240 not found: ID does not exist" containerID="ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240"
Oct 30 12:58:27.146067 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.146026    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240"} err="failed to get container status \"ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240\": rpc error: code = NotFound desc = could not find container \"ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240\": container with ID starting with ee6c3de75276ded134788f04f6315aeec3d63bc22f46dfb8a81f1a29c1a1f240 not found: ID does not exist"
Oct 30 12:58:27.146067 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.146062    5781 scope.go:122] "RemoveContainer" containerID="8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b"
Oct 30 12:58:27.146280 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:27.146260    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b\": container with ID starting with 8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b not found: ID does not exist" containerID="8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b"
Oct 30 12:58:27.146343 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.146304    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b"} err="failed to get container status \"8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b\": rpc error: code = NotFound desc = could not find container \"8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b\": container with ID starting with 8ca316107b9d52876d18c52deda60b51d0778d0f434b0bdb2c413fa58845bd2b not found: ID does not exist"
Oct 30 12:58:27.146343 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.146321    5781 scope.go:122] "RemoveContainer" containerID="b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069"
Oct 30 12:58:27.146546 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:27.146530    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069\": container with ID starting with b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069 not found: ID does not exist" containerID="b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069"
Oct 30 12:58:27.146587 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.146549    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069"} err="failed to get container status \"b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069\": rpc error: code = NotFound desc = could not find container \"b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069\": container with ID starting with b654067fbaab44ec9618cdb38685efeebc30712f6c095699ee3c0a64b66d2069 not found: ID does not exist"
Oct 30 12:58:27.146587 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.146562    5781 scope.go:122] "RemoveContainer" containerID="4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3"
Oct 30 12:58:27.146759 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:27.146744    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3\": container with ID starting with 4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3 not found: ID does not exist" containerID="4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3"
Oct 30 12:58:27.146800 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.146763    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3"} err="failed to get container status \"4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3\": rpc error: code = NotFound desc = could not find container \"4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3\": container with ID starting with 4fe73878fa12a00968de701bfc2f791aa19808776521c63957e75daa55782ec3 not found: ID does not exist"
Oct 30 12:58:27.146800 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.146776    5781 scope.go:122] "RemoveContainer" containerID="308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a"
Oct 30 12:58:27.147011 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:27.146996    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a\": container with ID starting with 308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a not found: ID does not exist" containerID="308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a"
Oct 30 12:58:27.147055 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.147013    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a"} err="failed to get container status \"308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a\": rpc error: code = NotFound desc = could not find container \"308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a\": container with ID starting with 308e5a4b71665da9d92b8285714f8e8b356006e1426c9d3f838e8ae77472f11a not found: ID does not exist"
Oct 30 12:58:27.147055 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.147024    5781 scope.go:122] "RemoveContainer" containerID="cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb"
Oct 30 12:58:27.147214 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:27.147193    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb\": container with ID starting with cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb not found: ID does not exist" containerID="cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb"
Oct 30 12:58:27.147253 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.147210    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb"} err="failed to get container status \"cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb\": rpc error: code = NotFound desc = could not find container \"cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb\": container with ID starting with cbb80b7dd1a137ea9bcdfadc5073784e55a207a5d67cc23902597a0f4dfab8bb not found: ID does not exist"
Oct 30 12:58:27.147253 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.147220    5781 scope.go:122] "RemoveContainer" containerID="7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f"
Oct 30 12:58:27.147442 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:27.147424    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f\": container with ID starting with 7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f not found: ID does not exist" containerID="7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f"
Oct 30 12:58:27.147484 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.147447    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f"} err="failed to get container status \"7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f\": rpc error: code = NotFound desc = could not find container \"7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f\": container with ID starting with 7ef5f8fd3d926aa77ee86ff05b33d046e097d302cf350ecfffaf27065031e85f not found: ID does not exist"
Oct 30 12:58:27.149308 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.149272    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 12:58:27.160831 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.160810    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 12:58:27.185364 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185328    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 12:58:27.185615 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185603    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="alertmanager"
Oct 30 12:58:27.185659 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185617    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="alertmanager"
Oct 30 12:58:27.185659 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185633    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="config-reloader"
Oct 30 12:58:27.185659 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185637    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="config-reloader"
Oct 30 12:58:27.185659 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185645    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy"
Oct 30 12:58:27.185659 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185650    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy"
Oct 30 12:58:27.185659 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185661    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy-web"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185666    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy-web"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185675    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="init-config-reloader"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185680    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="init-config-reloader"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185691    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="prom-label-proxy"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185695    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="prom-label-proxy"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185708    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy-metric"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185712    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy-metric"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185761    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="config-reloader"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185770    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185776    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy-web"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185782    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="init-config-reloader"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185788    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="prom-label-proxy"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185796    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="kube-rbac-proxy-metric"
Oct 30 12:58:27.185828 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.185801    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" containerName="alertmanager"
Oct 30 12:58:27.189710 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.189688    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.204307 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.204256    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 12:58:27.204447 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.204335    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 12:58:27.204514 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.204499    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 12:58:27.204573 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.204528    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-b82bm\""
Oct 30 12:58:27.204573 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.204529    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 12:58:27.204573 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.204570    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 12:58:27.204764 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.204749    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 12:58:27.204826 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.204752    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 12:58:27.204826 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.204767    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 12:58:27.216853 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.216831    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 12:58:27.220246 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.220221    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 12:58:27.238605 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.238567    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9618051c-041c-4f29-8555-01014e22b415-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.238605 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.238609    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.238804 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.238630    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.238804 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.238756    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.238873 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.238827    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9618051c-041c-4f29-8555-01014e22b415-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.238873 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.238854    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-config-volume\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.238931 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.238884    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.238967 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.238957    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.239036 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.239023    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5ptz\" (UniqueName: \"kubernetes.io/projected/9618051c-041c-4f29-8555-01014e22b415-kube-api-access-h5ptz\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.239067 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.239043    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-web-config\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.239117 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.239100    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9618051c-041c-4f29-8555-01014e22b415-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.239173 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.239161    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9618051c-041c-4f29-8555-01014e22b415-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.239209 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.239185    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9618051c-041c-4f29-8555-01014e22b415-config-out\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.339838 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.339802    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9618051c-041c-4f29-8555-01014e22b415-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340040 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.339848    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9618051c-041c-4f29-8555-01014e22b415-config-out\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340040 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.339883    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9618051c-041c-4f29-8555-01014e22b415-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340040 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.339913    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340040 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.339936    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340040 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.339961    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340040 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.339989    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9618051c-041c-4f29-8555-01014e22b415-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340040 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.340021    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-config-volume\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340462 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.340055    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340462 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.340085    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340462 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.340125    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5ptz\" (UniqueName: \"kubernetes.io/projected/9618051c-041c-4f29-8555-01014e22b415-kube-api-access-h5ptz\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340462 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.340149    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-web-config\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340462 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.340181    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9618051c-041c-4f29-8555-01014e22b415-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340462 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.340338    5781 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 12:58:27.340781 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.340646    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9618051c-041c-4f29-8555-01014e22b415-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.340781 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.340733    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9618051c-041c-4f29-8555-01014e22b415-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.341132 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.341078    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9618051c-041c-4f29-8555-01014e22b415-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.343059 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.343033    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9618051c-041c-4f29-8555-01014e22b415-config-out\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.343244 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.343211    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.343476 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.343460    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-web-config\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.343577 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.343552    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9618051c-041c-4f29-8555-01014e22b415-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.343668 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.343650    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.343815 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.343800    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.343987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.343965    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-config-volume\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.344466 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.344450    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.344526 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.344508    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9618051c-041c-4f29-8555-01014e22b415-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.364203 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.364137    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5ptz\" (UniqueName: \"kubernetes.io/projected/9618051c-041c-4f29-8555-01014e22b415-kube-api-access-h5ptz\") pod \"alertmanager-main-0\" (UID: \"9618051c-041c-4f29-8555-01014e22b415\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.514630 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.514598    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 12:58:27.851758 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:27.851729    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 12:58:27.856914 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:58:27.856882    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9618051c_041c_4f29_8555_01014e22b415.slice/crio-f4b9fffa9fedbd4d91b14bef622b9067312e53c1a2257a4f6ad85b94f0694779 WatchSource:0}: Error finding container f4b9fffa9fedbd4d91b14bef622b9067312e53c1a2257a4f6ad85b94f0694779: Status 404 returned error can't find the container with id f4b9fffa9fedbd4d91b14bef622b9067312e53c1a2257a4f6ad85b94f0694779
Oct 30 12:58:28.102226 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:28.102139    5781 generic.go:356] "Generic (PLEG): container finished" podID="9618051c-041c-4f29-8555-01014e22b415" containerID="f9e4fb2df4ccc5895b2ff6b4b51a4e89dae061c8a23bd892f50243335c77e403" exitCode=0
Oct 30 12:58:28.102368 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:28.102236    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9618051c-041c-4f29-8555-01014e22b415","Type":"ContainerDied","Data":"f9e4fb2df4ccc5895b2ff6b4b51a4e89dae061c8a23bd892f50243335c77e403"}
Oct 30 12:58:28.102368 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:28.102331    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9618051c-041c-4f29-8555-01014e22b415","Type":"ContainerStarted","Data":"f4b9fffa9fedbd4d91b14bef622b9067312e53c1a2257a4f6ad85b94f0694779"}
Oct 30 12:58:28.838686 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:28.838652    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0855ce73-dcfd-4398-b658-301a73ea990d" path="/var/lib/kubelet/pods/0855ce73-dcfd-4398-b658-301a73ea990d/volumes"
Oct 30 12:58:29.109318 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.109221    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9618051c-041c-4f29-8555-01014e22b415","Type":"ContainerStarted","Data":"6d069047aac965d75025e593b30446501f7ac378c1c26a6860cd79a6e714d414"}
Oct 30 12:58:29.109318 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.109305    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9618051c-041c-4f29-8555-01014e22b415","Type":"ContainerStarted","Data":"b6639ec943f39f69f3437cbbc9ea2815f8becc2eb60299b9ab8138544d18993d"}
Oct 30 12:58:29.109700 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.109334    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9618051c-041c-4f29-8555-01014e22b415","Type":"ContainerStarted","Data":"e6d6a4f6e56d9253a8d3b53f413461ade3b0e11bc4c4a42a6df9890d807a3cd9"}
Oct 30 12:58:29.109700 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.109359    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9618051c-041c-4f29-8555-01014e22b415","Type":"ContainerStarted","Data":"acdba4ae97c251670d7a3552b1002e697ac0708fe295a2307264af4460fa4928"}
Oct 30 12:58:29.109700 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.109392    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9618051c-041c-4f29-8555-01014e22b415","Type":"ContainerStarted","Data":"16019899c8e9b13c33b5763587a955d121b8bb5895aa5b46908d67994be897b9"}
Oct 30 12:58:29.109700 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.109425    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9618051c-041c-4f29-8555-01014e22b415","Type":"ContainerStarted","Data":"f6027e5536fae04b40ea744fef9fd972b477b4575d460d68c3401b5069703cc5"}
Oct 30 12:58:29.174571 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.174514    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.165374275 podStartE2EDuration="2.174500589s" totalImagesPullingTime="0s" totalInitContainerRuntime="9.126314ms" isStatefulPod=false podCreationTimestamp="2026-10-30 12:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:58:29.17344881 +0000 UTC m=+46.470666352" watchObservedRunningTime="2026-10-30 12:58:29.174500589 +0000 UTC m=+46.471718129"
Oct 30 12:58:29.458653 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.458619    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-bf446898-96f59"]
Oct 30 12:58:29.465376 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.465346    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.479631 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.479607    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 12:58:29.479743 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.479701    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 12:58:29.499379 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.499353    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bf446898-96f59"]
Oct 30 12:58:29.540140 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.540106    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 12:58:29.552557 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.552520    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 12:58:29.559720 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.559680    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49051fc-774c-485d-b50b-2486eaccd550-serving-certs-ca-bundle\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.559798 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.559752    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-telemeter-client-tls\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.559835 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.559820    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tgms\" (UniqueName: \"kubernetes.io/projected/b49051fc-774c-485d-b50b-2486eaccd550-kube-api-access-4tgms\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.559869 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.559844    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49051fc-774c-485d-b50b-2486eaccd550-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.559905 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.559872    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-federate-client-tls\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.559905 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.559894    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-secret-telemeter-client\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.559969 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.559931    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b49051fc-774c-485d-b50b-2486eaccd550-metrics-client-ca\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.560029 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.560012    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.580069 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.580038    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 12:58:29.580069 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.580054    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 12:58:29.580221 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.580146    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-k5gf6\""
Oct 30 12:58:29.661353 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.661322    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tgms\" (UniqueName: \"kubernetes.io/projected/b49051fc-774c-485d-b50b-2486eaccd550-kube-api-access-4tgms\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.661495 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.661357    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49051fc-774c-485d-b50b-2486eaccd550-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.661495 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.661461    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-federate-client-tls\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.661646 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.661626    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-secret-telemeter-client\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.661684 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.661663    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b49051fc-774c-485d-b50b-2486eaccd550-metrics-client-ca\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.661734 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.661718    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.661789 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.661753    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49051fc-774c-485d-b50b-2486eaccd550-serving-certs-ca-bundle\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.661845 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.661829    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-telemeter-client-tls\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.662454 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.662421    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b49051fc-774c-485d-b50b-2486eaccd550-metrics-client-ca\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.662578 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.662451    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49051fc-774c-485d-b50b-2486eaccd550-telemeter-trusted-ca-bundle\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.662578 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.662556    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b49051fc-774c-485d-b50b-2486eaccd550-serving-certs-ca-bundle\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.664107 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.664086    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.664199 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.664143    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-federate-client-tls\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.665044 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.665022    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-secret-telemeter-client\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.665121 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.665072    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b49051fc-774c-485d-b50b-2486eaccd550-telemeter-client-tls\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.683568 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.683546    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tgms\" (UniqueName: \"kubernetes.io/projected/b49051fc-774c-485d-b50b-2486eaccd550-kube-api-access-4tgms\") pod \"telemeter-client-bf446898-96f59\" (UID: \"b49051fc-774c-485d-b50b-2486eaccd550\") " pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:29.783426 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:29.783353    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-bf446898-96f59"
Oct 30 12:58:30.125824 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.125754    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-bf446898-96f59"]
Oct 30 12:58:30.131918 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:58:30.131884    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb49051fc_774c_485d_b50b_2486eaccd550.slice/crio-e78cc321abc1152e58b75e4bcb69de9176e5cead28c5acf312955cd1fda4ee14 WatchSource:0}: Error finding container e78cc321abc1152e58b75e4bcb69de9176e5cead28c5acf312955cd1fda4ee14: Status 404 returned error can't find the container with id e78cc321abc1152e58b75e4bcb69de9176e5cead28c5acf312955cd1fda4ee14
Oct 30 12:58:30.149369 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.142784    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 12:58:30.149369 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.143105    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="prometheus" containerID="cri-o://2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8" gracePeriod=600
Oct 30 12:58:30.149369 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.143319    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy" containerID="cri-o://e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a" gracePeriod=600
Oct 30 12:58:30.149369 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.143416    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy-web" containerID="cri-o://2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98" gracePeriod=600
Oct 30 12:58:30.149369 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.143508    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="thanos-sidecar" containerID="cri-o://51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7" gracePeriod=600
Oct 30 12:58:30.149369 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.143533    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751" gracePeriod=600
Oct 30 12:58:30.149369 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.143577    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="config-reloader" containerID="cri-o://423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c" gracePeriod=600
Oct 30 12:58:30.400511 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.400488    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:30.472242 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.472211    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.472778 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.472629    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.473006 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.472711    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.473170 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.472976    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.473408 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.473344    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.473629 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.473580    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.473858 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.473784    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474010 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.473969    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474311 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474183    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474311 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474230    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474311 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474270    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474919 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474635    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474919 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474681    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474919 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474716    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474919 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474753    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474919 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474778    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474919 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474815    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.474919 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.474874    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.475400 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.475366    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.475555 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.475511    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46\") pod \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\" (UID: \"221f1eb2-45dc-4f3e-b7d3-8487549100b4\") "
Oct 30 12:58:30.476779 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.476751    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:30.477073 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.477045    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:30.477707 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.477684    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:30.478081 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.478057    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:30.478984 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.478577    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:30.479277 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.479253    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:30.479436 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.479364    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:58:30.485528 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.485498    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:30.485683 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.485652    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:30.485840 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.485820    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:30.486461 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.486069    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "kube-api-access-qgd46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:58:30.486461 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.486124    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:30.486461 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.486208    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:30.486461 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.486330    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:30.486461 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.486437    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:30.487932 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.487900    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:58:30.488300 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.488268    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:58:30.488987 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.488966    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:30.489086 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.489047    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:30.495859 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.495836    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config" pod "221f1eb2-45dc-4f3e-b7d3-8487549100b4" (UID: "221f1eb2-45dc-4f3e-b7d3-8487549100b4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:30.577737 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577695    5781 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577737 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577736    5781 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-web-config\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577752    5781 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577767    5781 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577780    5781 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config-out\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577797    5781 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577812    5781 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-kube-rbac-proxy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577826    5781 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-config\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577839    5781 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-grpc-tls\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577887    5781 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577902    5781 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577916    5781 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-tls-assets\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577929    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgd46\" (UniqueName: \"kubernetes.io/projected/221f1eb2-45dc-4f3e-b7d3-8487549100b4-kube-api-access-qgd46\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577943    5781 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-metrics-client-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.577975 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577957    5781 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.578553 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.577971    5781 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.578553 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.578006    5781 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/221f1eb2-45dc-4f3e-b7d3-8487549100b4-prometheus-k8s-db\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.578553 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.578022    5781 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.578553 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.578036    5781 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/221f1eb2-45dc-4f3e-b7d3-8487549100b4-secret-metrics-client-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:30.578553 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:30.578052    5781 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/221f1eb2-45dc-4f3e-b7d3-8487549100b4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120695    5781 generic.go:356] "Generic (PLEG): container finished" podID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerID="5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751" exitCode=0
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120725    5781 generic.go:356] "Generic (PLEG): container finished" podID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerID="e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a" exitCode=0
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120735    5781 generic.go:356] "Generic (PLEG): container finished" podID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerID="2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98" exitCode=0
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120744    5781 generic.go:356] "Generic (PLEG): container finished" podID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerID="51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7" exitCode=0
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120752    5781 generic.go:356] "Generic (PLEG): container finished" podID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerID="423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c" exitCode=0
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120760    5781 generic.go:356] "Generic (PLEG): container finished" podID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerID="2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8" exitCode=0
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120778    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerDied","Data":"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"}
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120810    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120877    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerDied","Data":"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"}
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120892    5781 scope.go:122] "RemoveContainer" containerID="5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120923    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerDied","Data":"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"}
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.120967    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerDied","Data":"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"}
Oct 30 12:58:31.121022 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.121008    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerDied","Data":"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"}
Oct 30 12:58:31.121716 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.121049    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerDied","Data":"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"}
Oct 30 12:58:31.121716 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.121094    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"221f1eb2-45dc-4f3e-b7d3-8487549100b4","Type":"ContainerDied","Data":"e93a9a65d105becd00d0094f7a53873f2c021457989e5d1b00755c57d644e8e6"}
Oct 30 12:58:31.122700 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.122427    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bf446898-96f59" event={"ID":"b49051fc-774c-485d-b50b-2486eaccd550","Type":"ContainerStarted","Data":"e78cc321abc1152e58b75e4bcb69de9176e5cead28c5acf312955cd1fda4ee14"}
Oct 30 12:58:31.129187 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.129169    5781 scope.go:122] "RemoveContainer" containerID="e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"
Oct 30 12:58:31.137600 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.137583    5781 scope.go:122] "RemoveContainer" containerID="2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"
Oct 30 12:58:31.146776 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.146743    5781 scope.go:122] "RemoveContainer" containerID="51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"
Oct 30 12:58:31.155180 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.155160    5781 scope.go:122] "RemoveContainer" containerID="423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"
Oct 30 12:58:31.162369 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.162349    5781 scope.go:122] "RemoveContainer" containerID="2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"
Oct 30 12:58:31.170071 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.170054    5781 scope.go:122] "RemoveContainer" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"
Oct 30 12:58:31.177259 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.177190    5781 scope.go:122] "RemoveContainer" containerID="5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"
Oct 30 12:58:31.177635 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:31.177614    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": container with ID starting with 5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751 not found: ID does not exist" containerID="5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"
Oct 30 12:58:31.177732 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.177646    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"} err="failed to get container status \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": rpc error: code = NotFound desc = could not find container \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": container with ID starting with 5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751 not found: ID does not exist"
Oct 30 12:58:31.177732 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.177663    5781 scope.go:122] "RemoveContainer" containerID="e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"
Oct 30 12:58:31.177977 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:31.177944    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": container with ID starting with e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a not found: ID does not exist" containerID="e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"
Oct 30 12:58:31.178074 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.177974    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"} err="failed to get container status \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": rpc error: code = NotFound desc = could not find container \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": container with ID starting with e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a not found: ID does not exist"
Oct 30 12:58:31.178074 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.177992    5781 scope.go:122] "RemoveContainer" containerID="2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"
Oct 30 12:58:31.178319 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:31.178279    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": container with ID starting with 2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98 not found: ID does not exist" containerID="2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"
Oct 30 12:58:31.178412 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.178324    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"} err="failed to get container status \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": rpc error: code = NotFound desc = could not find container \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": container with ID starting with 2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98 not found: ID does not exist"
Oct 30 12:58:31.178412 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.178340    5781 scope.go:122] "RemoveContainer" containerID="51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"
Oct 30 12:58:31.178729 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:31.178706    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": container with ID starting with 51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7 not found: ID does not exist" containerID="51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"
Oct 30 12:58:31.178821 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.178733    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"} err="failed to get container status \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": rpc error: code = NotFound desc = could not find container \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": container with ID starting with 51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7 not found: ID does not exist"
Oct 30 12:58:31.178821 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.178754    5781 scope.go:122] "RemoveContainer" containerID="423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"
Oct 30 12:58:31.179042 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:31.179018    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": container with ID starting with 423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c not found: ID does not exist" containerID="423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"
Oct 30 12:58:31.179109 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.179050    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"} err="failed to get container status \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": rpc error: code = NotFound desc = could not find container \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": container with ID starting with 423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c not found: ID does not exist"
Oct 30 12:58:31.179109 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.179071    5781 scope.go:122] "RemoveContainer" containerID="2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"
Oct 30 12:58:31.179402 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:31.179378    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": container with ID starting with 2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8 not found: ID does not exist" containerID="2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"
Oct 30 12:58:31.179463 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.179410    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"} err="failed to get container status \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": rpc error: code = NotFound desc = could not find container \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": container with ID starting with 2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8 not found: ID does not exist"
Oct 30 12:58:31.179463 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.179430    5781 scope.go:122] "RemoveContainer" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"
Oct 30 12:58:31.179735 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:31.179702    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": container with ID starting with 41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f not found: ID does not exist" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"
Oct 30 12:58:31.179832 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.179746    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"} err="failed to get container status \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": rpc error: code = NotFound desc = could not find container \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": container with ID starting with 41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f not found: ID does not exist"
Oct 30 12:58:31.179832 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.179761    5781 scope.go:122] "RemoveContainer" containerID="5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"
Oct 30 12:58:31.180078 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.180043    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"} err="failed to get container status \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": rpc error: code = NotFound desc = could not find container \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": container with ID starting with 5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751 not found: ID does not exist"
Oct 30 12:58:31.180178 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.180080    5781 scope.go:122] "RemoveContainer" containerID="e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"
Oct 30 12:58:31.180398 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.180376    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"} err="failed to get container status \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": rpc error: code = NotFound desc = could not find container \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": container with ID starting with e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a not found: ID does not exist"
Oct 30 12:58:31.180475 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.180399    5781 scope.go:122] "RemoveContainer" containerID="2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"
Oct 30 12:58:31.180704 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.180677    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"} err="failed to get container status \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": rpc error: code = NotFound desc = could not find container \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": container with ID starting with 2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98 not found: ID does not exist"
Oct 30 12:58:31.180783 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.180704    5781 scope.go:122] "RemoveContainer" containerID="51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"
Oct 30 12:58:31.181007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.180983    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"} err="failed to get container status \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": rpc error: code = NotFound desc = could not find container \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": container with ID starting with 51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7 not found: ID does not exist"
Oct 30 12:58:31.181007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.181007    5781 scope.go:122] "RemoveContainer" containerID="423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"
Oct 30 12:58:31.181327 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.181282    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"} err="failed to get container status \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": rpc error: code = NotFound desc = could not find container \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": container with ID starting with 423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c not found: ID does not exist"
Oct 30 12:58:31.181419 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.181329    5781 scope.go:122] "RemoveContainer" containerID="2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"
Oct 30 12:58:31.181647 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.181619    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"} err="failed to get container status \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": rpc error: code = NotFound desc = could not find container \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": container with ID starting with 2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8 not found: ID does not exist"
Oct 30 12:58:31.181647 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.181637    5781 scope.go:122] "RemoveContainer" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"
Oct 30 12:58:31.181893 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.181861    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"} err="failed to get container status \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": rpc error: code = NotFound desc = could not find container \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": container with ID starting with 41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f not found: ID does not exist"
Oct 30 12:58:31.181893 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.181886    5781 scope.go:122] "RemoveContainer" containerID="5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"
Oct 30 12:58:31.182302 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.182259    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"} err="failed to get container status \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": rpc error: code = NotFound desc = could not find container \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": container with ID starting with 5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751 not found: ID does not exist"
Oct 30 12:58:31.182394 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.182307    5781 scope.go:122] "RemoveContainer" containerID="e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"
Oct 30 12:58:31.182547 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.182523    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"} err="failed to get container status \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": rpc error: code = NotFound desc = could not find container \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": container with ID starting with e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a not found: ID does not exist"
Oct 30 12:58:31.182547 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.182546    5781 scope.go:122] "RemoveContainer" containerID="2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"
Oct 30 12:58:31.182809 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.182789    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"} err="failed to get container status \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": rpc error: code = NotFound desc = could not find container \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": container with ID starting with 2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98 not found: ID does not exist"
Oct 30 12:58:31.182885 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.182808    5781 scope.go:122] "RemoveContainer" containerID="51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"
Oct 30 12:58:31.183119 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.183091    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"} err="failed to get container status \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": rpc error: code = NotFound desc = could not find container \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": container with ID starting with 51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7 not found: ID does not exist"
Oct 30 12:58:31.183119 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.183117    5781 scope.go:122] "RemoveContainer" containerID="423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"
Oct 30 12:58:31.183417 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.183399    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"} err="failed to get container status \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": rpc error: code = NotFound desc = could not find container \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": container with ID starting with 423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c not found: ID does not exist"
Oct 30 12:58:31.183490 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.183418    5781 scope.go:122] "RemoveContainer" containerID="2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"
Oct 30 12:58:31.183655 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.183635    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"} err="failed to get container status \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": rpc error: code = NotFound desc = could not find container \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": container with ID starting with 2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8 not found: ID does not exist"
Oct 30 12:58:31.183655 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.183653    5781 scope.go:122] "RemoveContainer" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"
Oct 30 12:58:31.183958 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.183866    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"} err="failed to get container status \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": rpc error: code = NotFound desc = could not find container \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": container with ID starting with 41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f not found: ID does not exist"
Oct 30 12:58:31.183958 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.183897    5781 scope.go:122] "RemoveContainer" containerID="5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"
Oct 30 12:58:31.184165 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.184144    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"} err="failed to get container status \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": rpc error: code = NotFound desc = could not find container \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": container with ID starting with 5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751 not found: ID does not exist"
Oct 30 12:58:31.184232 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.184166    5781 scope.go:122] "RemoveContainer" containerID="e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"
Oct 30 12:58:31.184439 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.184421    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"} err="failed to get container status \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": rpc error: code = NotFound desc = could not find container \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": container with ID starting with e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a not found: ID does not exist"
Oct 30 12:58:31.184439 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.184439    5781 scope.go:122] "RemoveContainer" containerID="2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"
Oct 30 12:58:31.184676 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.184658    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"} err="failed to get container status \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": rpc error: code = NotFound desc = could not find container \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": container with ID starting with 2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98 not found: ID does not exist"
Oct 30 12:58:31.184676 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.184675    5781 scope.go:122] "RemoveContainer" containerID="51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"
Oct 30 12:58:31.184922 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.184900    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"} err="failed to get container status \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": rpc error: code = NotFound desc = could not find container \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": container with ID starting with 51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7 not found: ID does not exist"
Oct 30 12:58:31.184999 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.184922    5781 scope.go:122] "RemoveContainer" containerID="423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"
Oct 30 12:58:31.185194 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.185169    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"} err="failed to get container status \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": rpc error: code = NotFound desc = could not find container \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": container with ID starting with 423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c not found: ID does not exist"
Oct 30 12:58:31.185271 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.185195    5781 scope.go:122] "RemoveContainer" containerID="2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"
Oct 30 12:58:31.186399 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.186380    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 12:58:31.186399 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.186389    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"} err="failed to get container status \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": rpc error: code = NotFound desc = could not find container \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": container with ID starting with 2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8 not found: ID does not exist"
Oct 30 12:58:31.186531 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.186408    5781 scope.go:122] "RemoveContainer" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"
Oct 30 12:58:31.187042 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.187016    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"} err="failed to get container status \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": rpc error: code = NotFound desc = could not find container \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": container with ID starting with 41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f not found: ID does not exist"
Oct 30 12:58:31.187123 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.187042    5781 scope.go:122] "RemoveContainer" containerID="5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"
Oct 30 12:58:31.187359 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.187317    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"} err="failed to get container status \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": rpc error: code = NotFound desc = could not find container \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": container with ID starting with 5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751 not found: ID does not exist"
Oct 30 12:58:31.187359 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.187343    5781 scope.go:122] "RemoveContainer" containerID="e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"
Oct 30 12:58:31.187599 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.187578    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"} err="failed to get container status \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": rpc error: code = NotFound desc = could not find container \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": container with ID starting with e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a not found: ID does not exist"
Oct 30 12:58:31.187685 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.187600    5781 scope.go:122] "RemoveContainer" containerID="2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"
Oct 30 12:58:31.187897 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.187873    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"} err="failed to get container status \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": rpc error: code = NotFound desc = could not find container \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": container with ID starting with 2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98 not found: ID does not exist"
Oct 30 12:58:31.187897 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.187891    5781 scope.go:122] "RemoveContainer" containerID="51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"
Oct 30 12:58:31.188161 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.188142    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"} err="failed to get container status \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": rpc error: code = NotFound desc = could not find container \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": container with ID starting with 51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7 not found: ID does not exist"
Oct 30 12:58:31.188215 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.188167    5781 scope.go:122] "RemoveContainer" containerID="423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"
Oct 30 12:58:31.188387 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.188369    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"} err="failed to get container status \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": rpc error: code = NotFound desc = could not find container \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": container with ID starting with 423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c not found: ID does not exist"
Oct 30 12:58:31.188450 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.188387    5781 scope.go:122] "RemoveContainer" containerID="2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"
Oct 30 12:58:31.188618 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.188599    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"} err="failed to get container status \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": rpc error: code = NotFound desc = could not find container \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": container with ID starting with 2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8 not found: ID does not exist"
Oct 30 12:58:31.188684 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.188617    5781 scope.go:122] "RemoveContainer" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"
Oct 30 12:58:31.188869 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.188849    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"} err="failed to get container status \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": rpc error: code = NotFound desc = could not find container \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": container with ID starting with 41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f not found: ID does not exist"
Oct 30 12:58:31.188924 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.188869    5781 scope.go:122] "RemoveContainer" containerID="5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"
Oct 30 12:58:31.189165 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.189143    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751"} err="failed to get container status \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": rpc error: code = NotFound desc = could not find container \"5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751\": container with ID starting with 5e27d5df3f3c47baacaf9f33a98853273b4f5fb28834a0518ced5e4158afb751 not found: ID does not exist"
Oct 30 12:58:31.189246 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.189166    5781 scope.go:122] "RemoveContainer" containerID="e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"
Oct 30 12:58:31.189489 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.189469    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a"} err="failed to get container status \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": rpc error: code = NotFound desc = could not find container \"e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a\": container with ID starting with e86965312808ed9c1a0b2789e9fb65c6baa2619b96c9c2fa8ac19d0f1f1ece3a not found: ID does not exist"
Oct 30 12:58:31.189489 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.189489    5781 scope.go:122] "RemoveContainer" containerID="2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"
Oct 30 12:58:31.189760 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.189734    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98"} err="failed to get container status \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": rpc error: code = NotFound desc = could not find container \"2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98\": container with ID starting with 2b28a125d9c6dcc265f0504995b0234ba7cf57f1f205434a4a3140cd13e03c98 not found: ID does not exist"
Oct 30 12:58:31.189842 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.189762    5781 scope.go:122] "RemoveContainer" containerID="51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"
Oct 30 12:58:31.190007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.189989    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7"} err="failed to get container status \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": rpc error: code = NotFound desc = could not find container \"51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7\": container with ID starting with 51ec8152a48f4a71bad40a8b7d699624e1c98712cb893bdc9ada63804bfca7d7 not found: ID does not exist"
Oct 30 12:58:31.190007 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.190006    5781 scope.go:122] "RemoveContainer" containerID="423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"
Oct 30 12:58:31.190244 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.190225    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c"} err="failed to get container status \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": rpc error: code = NotFound desc = could not find container \"423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c\": container with ID starting with 423e286b5b5c1c46a1f64bccb479dfb96acabfeff268af5dc79d19c7ae867e6c not found: ID does not exist"
Oct 30 12:58:31.190335 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.190245    5781 scope.go:122] "RemoveContainer" containerID="2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"
Oct 30 12:58:31.190564 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.190544    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8"} err="failed to get container status \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": rpc error: code = NotFound desc = could not find container \"2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8\": container with ID starting with 2ee2f9488cac24aa61a2c6e8260cc198e902a1d72708f47250ac5b1ea3701bc8 not found: ID does not exist"
Oct 30 12:58:31.190617 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.190563    5781 scope.go:122] "RemoveContainer" containerID="41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"
Oct 30 12:58:31.190796 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.190778    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f"} err="failed to get container status \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": rpc error: code = NotFound desc = could not find container \"41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f\": container with ID starting with 41bb7e4bc644d4da6e4f5fcaf8d95ea28cd4d344e7cee42b68ce332a5f12233f not found: ID does not exist"
Oct 30 12:58:31.198348 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.198324    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 12:58:31.224358 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224335    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 12:58:31.224709 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224692    5781 cpu_manager.go:414] "containerMap: removing container" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="prometheus"
Oct 30 12:58:31.224801 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224710    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="prometheus"
Oct 30 12:58:31.224801 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224721    5781 cpu_manager.go:414] "containerMap: removing container" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy-web"
Oct 30 12:58:31.224801 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224726    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy-web"
Oct 30 12:58:31.224801 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224771    5781 cpu_manager.go:414] "containerMap: removing container" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy"
Oct 30 12:58:31.224801 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224779    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy"
Oct 30 12:58:31.224801 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224801    5781 cpu_manager.go:414] "containerMap: removing container" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="init-config-reloader"
Oct 30 12:58:31.225221 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224808    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="init-config-reloader"
Oct 30 12:58:31.225221 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224818    5781 cpu_manager.go:414] "containerMap: removing container" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="thanos-sidecar"
Oct 30 12:58:31.225221 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224845    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="thanos-sidecar"
Oct 30 12:58:31.225221 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224864    5781 cpu_manager.go:414] "containerMap: removing container" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy-thanos"
Oct 30 12:58:31.225221 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224871    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy-thanos"
Oct 30 12:58:31.225221 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224885    5781 cpu_manager.go:414] "containerMap: removing container" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="config-reloader"
Oct 30 12:58:31.225221 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224893    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="config-reloader"
Oct 30 12:58:31.225221 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.224999    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy"
Oct 30 12:58:31.225623 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.225016    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="thanos-sidecar"
Oct 30 12:58:31.225623 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.225263    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy-thanos"
Oct 30 12:58:31.225623 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.225279    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="config-reloader"
Oct 30 12:58:31.225623 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.225305    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="prometheus"
Oct 30 12:58:31.225623 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.225350    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" containerName="kube-rbac-proxy-web"
Oct 30 12:58:31.233014 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.232987    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.251991 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.251971    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 12:58:31.252834 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.252812    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 12:58:31.252834 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.252813    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 12:58:31.252979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.252814    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 12:58:31.252979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.252850    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 12:58:31.252979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.252854    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 12:58:31.252979 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.252919    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 12:58:31.253158 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.252980    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-82464\""
Oct 30 12:58:31.253158 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.253103    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 12:58:31.253158 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.253142    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 12:58:31.253158 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.253147    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 12:58:31.253468 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.253451    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 12:58:31.253528 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.253467    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 12:58:31.253528 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.253479    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cacau0ihglokf\""
Oct 30 12:58:31.265909 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.265888    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 12:58:31.265909 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.265928    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 12:58:31.266951 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.266935    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 12:58:31.283344 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283321    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283452 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283350    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283452 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283378    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283567 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283461    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283567 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283553    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-config\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283656 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283589    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283656 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283637    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283733 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283682    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283733 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283723    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5848b5af-edfa-4001-9c92-4cfa9f528d32-config-out\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283814 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283742    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-web-config\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283814 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283763    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283892 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283871    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5848b5af-edfa-4001-9c92-4cfa9f528d32-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283928 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283895    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.283928 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283922    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.284038 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283944    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trm2j\" (UniqueName: \"kubernetes.io/projected/5848b5af-edfa-4001-9c92-4cfa9f528d32-kube-api-access-trm2j\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.284038 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283960    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.284038 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.283979    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.284038 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.284000    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.284038 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.284025    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.284215 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.284042    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.384977 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.384899    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.384977 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.384940    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-config\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.384977 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.384970    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385257 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.384994    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385257 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385019    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385257 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385056    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5848b5af-edfa-4001-9c92-4cfa9f528d32-config-out\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385257 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385128    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-web-config\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385257 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385172    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385257 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385210    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5848b5af-edfa-4001-9c92-4cfa9f528d32-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385257 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385243    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385308    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385334    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trm2j\" (UniqueName: \"kubernetes.io/projected/5848b5af-edfa-4001-9c92-4cfa9f528d32-kube-api-access-trm2j\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385359    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385393    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385431    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385463    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385489    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385524    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385540    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.385626 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385555    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.386113 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.385675    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.386113 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.386065    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.386212 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.386179    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.388443 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.388154    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.388999 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.388975    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5848b5af-edfa-4001-9c92-4cfa9f528d32-config-out\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.389105 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.389088    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5848b5af-edfa-4001-9c92-4cfa9f528d32-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.389281 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.389262    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.389406 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.389385    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.389495 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.389476    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.389664 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.389640    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.389844 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.389826    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.390159 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.390121    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.390749 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.390697    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.390749 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.390724    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.390946 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.390848    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.392169 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.392145    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-web-config\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.392591 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.392568    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.393029 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.393008    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5848b5af-edfa-4001-9c92-4cfa9f528d32-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.394195 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.394177    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5848b5af-edfa-4001-9c92-4cfa9f528d32-config\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.412674 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.412651    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trm2j\" (UniqueName: \"kubernetes.io/projected/5848b5af-edfa-4001-9c92-4cfa9f528d32-kube-api-access-trm2j\") pod \"prometheus-k8s-0\" (UID: \"5848b5af-edfa-4001-9c92-4cfa9f528d32\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.560445 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.560410    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:31.896802 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:31.896723    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 12:58:31.902640 ip-10-0-130-180 kubenswrapper[5781]: W1030 12:58:31.902611    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5848b5af_edfa_4001_9c92_4cfa9f528d32.slice/crio-61ffee5142259036e89a2446d8e90bece5f266442573272fdcfd6149040997fd WatchSource:0}: Error finding container 61ffee5142259036e89a2446d8e90bece5f266442573272fdcfd6149040997fd: Status 404 returned error can't find the container with id 61ffee5142259036e89a2446d8e90bece5f266442573272fdcfd6149040997fd
Oct 30 12:58:31.998199 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:58:31.998167    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 12:58:32.127776 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.127749    5781 generic.go:356] "Generic (PLEG): container finished" podID="5848b5af-edfa-4001-9c92-4cfa9f528d32" containerID="3b6ef2a528cf2ae1b662ccd239f102e06111f7a8531eb31c9448ee15ada99d05" exitCode=0
Oct 30 12:58:32.127933 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.127843    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5848b5af-edfa-4001-9c92-4cfa9f528d32","Type":"ContainerDied","Data":"3b6ef2a528cf2ae1b662ccd239f102e06111f7a8531eb31c9448ee15ada99d05"}
Oct 30 12:58:32.127983 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.127937    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5848b5af-edfa-4001-9c92-4cfa9f528d32","Type":"ContainerStarted","Data":"61ffee5142259036e89a2446d8e90bece5f266442573272fdcfd6149040997fd"}
Oct 30 12:58:32.130085 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.130051    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bf446898-96f59" event={"ID":"b49051fc-774c-485d-b50b-2486eaccd550","Type":"ContainerStarted","Data":"19494af383287a8aab326394b6659bcdcc9cbce6ebca9e4d4141430448a76aa1"}
Oct 30 12:58:32.130434 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.130102    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bf446898-96f59" event={"ID":"b49051fc-774c-485d-b50b-2486eaccd550","Type":"ContainerStarted","Data":"7a30833551e52107c0d900e2a63e2f27510b66093da602d1f90417eece4b0353"}
Oct 30 12:58:32.130434 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.130126    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-bf446898-96f59" event={"ID":"b49051fc-774c-485d-b50b-2486eaccd550","Type":"ContainerStarted","Data":"8e691d8bca60492f64821c37215f9fb5c16c5b08ec2782cf34722a2f98c5cafe"}
Oct 30 12:58:32.244558 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.244518    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-bf446898-96f59" podStartSLOduration=1.761278317 podStartE2EDuration="3.244503626s" totalImagesPullingTime="1.483225309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:58:32.243614432 +0000 UTC m=+49.540831976" watchObservedRunningTime="2026-10-30 12:58:32.244503626 +0000 UTC m=+49.541721163"
Oct 30 12:58:32.630819 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.630788    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:32.811570 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.811478    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:32.841856 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:32.841825    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="221f1eb2-45dc-4f3e-b7d3-8487549100b4" path="/var/lib/kubelet/pods/221f1eb2-45dc-4f3e-b7d3-8487549100b4/volumes"
Oct 30 12:58:33.024139 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:33.024101    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:33.137824 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:33.137754    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5848b5af-edfa-4001-9c92-4cfa9f528d32","Type":"ContainerStarted","Data":"2906ad328adfc5d1967c21ad906365d556fc22c455832449196e0e814ffb953a"}
Oct 30 12:58:33.138216 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:33.137828    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5848b5af-edfa-4001-9c92-4cfa9f528d32","Type":"ContainerStarted","Data":"a7de265d25895610cb25bbc3d2318fdcc3be9950e33768e09e517afc1b3beaa7"}
Oct 30 12:58:33.138216 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:33.137856    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5848b5af-edfa-4001-9c92-4cfa9f528d32","Type":"ContainerStarted","Data":"2d4b84dc188376bb7cb45be2f5d99b62b35c82d199848bed88e3e6ae3a2509b6"}
Oct 30 12:58:33.138216 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:33.137878    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5848b5af-edfa-4001-9c92-4cfa9f528d32","Type":"ContainerStarted","Data":"86ebd8fb03d58fa3bd40409021cc01726365d8ab18a51a298d3b3b21852d5ea0"}
Oct 30 12:58:33.138216 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:33.137909    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5848b5af-edfa-4001-9c92-4cfa9f528d32","Type":"ContainerStarted","Data":"b2856ffa32712eea6a795de9e1ae3cfa09d83aca8b4426365520f0e493da1558"}
Oct 30 12:58:33.138216 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:33.137936    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5848b5af-edfa-4001-9c92-4cfa9f528d32","Type":"ContainerStarted","Data":"c16f2eab6a0af139a5e60426033289d5e92fc0dcf10cca6c63c9952d989b5685"}
Oct 30 12:58:33.197822 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:33.197770    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.181351599 podStartE2EDuration="2.197758449s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.40685ms" isStatefulPod=false podCreationTimestamp="2026-10-30 12:58:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:58:33.197211503 +0000 UTC m=+50.494429050" watchObservedRunningTime="2026-10-30 12:58:33.197758449 +0000 UTC m=+50.494975989"
Oct 30 12:58:36.560955 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:36.560911    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:58:54.174149 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:54.174094    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:58:54.323859 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:58:54.323819    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:59:01.995716 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:59:01.995674    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 12:59:31.366205 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:31.366132    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:59:31.381488 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:31.381466    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:59:31.576577 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:31.576551    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 12:59:31.995304 ip-10-0-130-180 kubenswrapper[5781]: E1030 12:59:31.995258    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 12:59:40.147936 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:40.147901    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:59:40.302049 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:40.302014    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:59:52.729547 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:52.729504    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:59:52.865067 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:52.865038    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:59:54.281030 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:54.280996    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:59:56.491475 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:56.491445    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 12:59:56.639930 ip-10-0-130-180 kubenswrapper[5781]: I1030 12:59:56.639901    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:00:01.996740 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:00:01.996705    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:00:06.199054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:00:06.199024    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:00:06.685474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:00:06.685443    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:00:07.749117 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:00:07.749083    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:00:09.860074 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:00:09.860037    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:00:10.004273 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:00:10.004244    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:00:10.186160 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:00:10.186133    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:00:10.406838 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:00:10.406806    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:00:31.997649 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:00:31.997605    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:00:54.422555 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:00:54.422482    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:00:59.283848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:00:59.283815    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:01:01.994829 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:01:01.994792    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:01:04.428156 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:01:04.428118    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:01:31.994549 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:01:31.994511    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:01:49.022480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:01:49.022447    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:01:50.606960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:01:50.606928    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:01:54.561747 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:01:54.561713    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:01:59.390047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:01:59.390013    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:01.995080 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:02:01.995045    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:02:03.508104 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:03.508070    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:04.554670 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:04.554623    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:07.959532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:07.959501    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:08.127040 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:08.127008    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:11.951940 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:11.951909    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:13.400180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:13.400097    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:15.466631 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:15.466597    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:15.651379 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:15.651353    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:20.195238 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:20.195206    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:20.387331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:20.387283    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:26.611247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:26.611183    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:26.923241 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:26.923212    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:28.691208 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:28.691178    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:29.015666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:29.015575    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:31.994497 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:02:31.994464    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:02:36.352503 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:36.352466    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:36.561351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:36.561318    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:37.064518 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.064484    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:37.446163 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.446131    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-qrptz"]
Oct 30 13:02:37.449598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.449576    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-qrptz"
Oct 30 13:02:37.463576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.463557    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 13:02:37.463675 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.463581    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 13:02:37.475120 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.475095    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-97zbp\""
Oct 30 13:02:37.475890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.475874    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-qrptz"]
Oct 30 13:02:37.579102 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.579072    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwtx8\" (UniqueName: \"kubernetes.io/projected/2d4e0b92-f40d-4d1f-a26b-5564fc265c13-kube-api-access-nwtx8\") pod \"cert-manager-597b95cb84-qrptz\" (UID: \"2d4e0b92-f40d-4d1f-a26b-5564fc265c13\") " pod="cert-manager/cert-manager-597b95cb84-qrptz"
Oct 30 13:02:37.579264 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.579182    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d4e0b92-f40d-4d1f-a26b-5564fc265c13-bound-sa-token\") pod \"cert-manager-597b95cb84-qrptz\" (UID: \"2d4e0b92-f40d-4d1f-a26b-5564fc265c13\") " pod="cert-manager/cert-manager-597b95cb84-qrptz"
Oct 30 13:02:37.620462 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.620437    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:37.679726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.679703    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d4e0b92-f40d-4d1f-a26b-5564fc265c13-bound-sa-token\") pod \"cert-manager-597b95cb84-qrptz\" (UID: \"2d4e0b92-f40d-4d1f-a26b-5564fc265c13\") " pod="cert-manager/cert-manager-597b95cb84-qrptz"
Oct 30 13:02:37.679817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.679742    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwtx8\" (UniqueName: \"kubernetes.io/projected/2d4e0b92-f40d-4d1f-a26b-5564fc265c13-kube-api-access-nwtx8\") pod \"cert-manager-597b95cb84-qrptz\" (UID: \"2d4e0b92-f40d-4d1f-a26b-5564fc265c13\") " pod="cert-manager/cert-manager-597b95cb84-qrptz"
Oct 30 13:02:37.703560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.703509    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2d4e0b92-f40d-4d1f-a26b-5564fc265c13-bound-sa-token\") pod \"cert-manager-597b95cb84-qrptz\" (UID: \"2d4e0b92-f40d-4d1f-a26b-5564fc265c13\") " pod="cert-manager/cert-manager-597b95cb84-qrptz"
Oct 30 13:02:37.703665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.703649    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwtx8\" (UniqueName: \"kubernetes.io/projected/2d4e0b92-f40d-4d1f-a26b-5564fc265c13-kube-api-access-nwtx8\") pod \"cert-manager-597b95cb84-qrptz\" (UID: \"2d4e0b92-f40d-4d1f-a26b-5564fc265c13\") " pod="cert-manager/cert-manager-597b95cb84-qrptz"
Oct 30 13:02:37.776691 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:37.776665    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-qrptz"
Oct 30 13:02:38.072485 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:38.072409    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-qrptz"]
Oct 30 13:02:38.077910 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:02:38.077883    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d4e0b92_f40d_4d1f_a26b_5564fc265c13.slice/crio-e704206b5c7112590f6418ac1e53616ee2857e92a157caa84a84a0f09da16667 WatchSource:0}: Error finding container e704206b5c7112590f6418ac1e53616ee2857e92a157caa84a84a0f09da16667: Status 404 returned error can't find the container with id e704206b5c7112590f6418ac1e53616ee2857e92a157caa84a84a0f09da16667
Oct 30 13:02:38.776542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:38.776481    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:38.929628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:38.929565    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:39.048894 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:39.048811    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-qrptz" event={"ID":"2d4e0b92-f40d-4d1f-a26b-5564fc265c13","Type":"ContainerStarted","Data":"e704206b5c7112590f6418ac1e53616ee2857e92a157caa84a84a0f09da16667"}
Oct 30 13:02:41.058343 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:41.058308    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-qrptz" event={"ID":"2d4e0b92-f40d-4d1f-a26b-5564fc265c13","Type":"ContainerStarted","Data":"badbcfcc322db7d7d39d1746643c427a30281afe0375b493a820a2118b501df3"}
Oct 30 13:02:41.097844 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:41.097757    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-qrptz" podStartSLOduration=1.339644163 podStartE2EDuration="4.09774454s" totalImagesPullingTime="2.758100377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:41.097172529 +0000 UTC m=+298.394390070" watchObservedRunningTime="2026-10-30 13:02:41.09774454 +0000 UTC m=+298.394962080"
Oct 30 13:02:42.862547 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:42.862510    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:02:42.865626 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:42.865601    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:02:47.090080 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:47.090043    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:47.323182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:47.323151    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:54.604795 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:54.604755    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:54.785586 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:54.785551    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:02:59.522557 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:02:59.522519    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:01.993986 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:03:01.993954    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:03:04.710257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:04.710219    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:05.127671 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.127585    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:05.243193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.243161    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"]
Oct 30 13:03:05.249338 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.249313    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.299404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.299380    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"]
Oct 30 13:03:05.309215 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.309196    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 30 13:03:05.309342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.309198    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 30 13:03:05.309342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.309216    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 30 13:03:05.309732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.309715    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 30 13:03:05.309774 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.309737    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 30 13:03:05.310145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.310133    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kwrjb\""
Oct 30 13:03:05.403057 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.403002    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b238bb20-f8b9-4381-a3f0-5da73b146ba9-cert\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.403057 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.403052    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvpgt\" (UniqueName: \"kubernetes.io/projected/b238bb20-f8b9-4381-a3f0-5da73b146ba9-kube-api-access-gvpgt\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.403205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.403125    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b238bb20-f8b9-4381-a3f0-5da73b146ba9-manager-config\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.403205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.403147    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b238bb20-f8b9-4381-a3f0-5da73b146ba9-metrics-cert\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.503695 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.503669    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b238bb20-f8b9-4381-a3f0-5da73b146ba9-cert\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.503802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.503714    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvpgt\" (UniqueName: \"kubernetes.io/projected/b238bb20-f8b9-4381-a3f0-5da73b146ba9-kube-api-access-gvpgt\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.503802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.503750    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b238bb20-f8b9-4381-a3f0-5da73b146ba9-manager-config\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.503802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.503776    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b238bb20-f8b9-4381-a3f0-5da73b146ba9-metrics-cert\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.504441 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.504423    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b238bb20-f8b9-4381-a3f0-5da73b146ba9-manager-config\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.506160 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.506139    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b238bb20-f8b9-4381-a3f0-5da73b146ba9-cert\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.506315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.506273    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b238bb20-f8b9-4381-a3f0-5da73b146ba9-metrics-cert\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.525943 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.525916    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:05.530710 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.530693    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvpgt\" (UniqueName: \"kubernetes.io/projected/b238bb20-f8b9-4381-a3f0-5da73b146ba9-kube-api-access-gvpgt\") pod \"lws-controller-manager-69565795b9-f6s4q\" (UID: \"b238bb20-f8b9-4381-a3f0-5da73b146ba9\") " pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.559794 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.559775    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:05.861704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.861672    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"]
Oct 30 13:03:05.867443 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:03:05.867413    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb238bb20_f8b9_4381_a3f0_5da73b146ba9.slice/crio-17ffc81180c051c53e8c6f306acb8ceabddf4651ca67617d3f606aba461da7c7 WatchSource:0}: Error finding container 17ffc81180c051c53e8c6f306acb8ceabddf4651ca67617d3f606aba461da7c7: Status 404 returned error can't find the container with id 17ffc81180c051c53e8c6f306acb8ceabddf4651ca67617d3f606aba461da7c7
Oct 30 13:03:05.869430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:05.869415    5781 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:03:06.147445 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:06.147372    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q" event={"ID":"b238bb20-f8b9-4381-a3f0-5da73b146ba9","Type":"ContainerStarted","Data":"17ffc81180c051c53e8c6f306acb8ceabddf4651ca67617d3f606aba461da7c7"}
Oct 30 13:03:09.161605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:09.161569    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q" event={"ID":"b238bb20-f8b9-4381-a3f0-5da73b146ba9","Type":"ContainerStarted","Data":"f843cb1377b583d1932a6d46c72299615675f0cec0edaf4318fc2a4bf1329976"}
Oct 30 13:03:09.161991 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:09.161649    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:09.207172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:09.207123    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q" podStartSLOduration=1.7178553349999999 podStartE2EDuration="4.207110029s" totalImagesPullingTime="2.489254694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:09.206860212 +0000 UTC m=+326.504077756" watchObservedRunningTime="2026-10-30 13:03:09.207110029 +0000 UTC m=+326.504327569"
Oct 30 13:03:12.261177 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:12.261143    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:12.450619 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:12.450583    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:20.167949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:20.167919    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-69565795b9-f6s4q"
Oct 30 13:03:27.621161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:27.621119    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:27.878060 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:27.877973    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:28.264311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.264254    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:28.433248 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.433220    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"]
Oct 30 13:03:28.436733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.436717    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.452507 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.452483    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 13:03:28.454867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.454847    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 13:03:28.464116 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.464097    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 13:03:28.464190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.464173    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 13:03:28.481539 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.481520    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"]
Oct 30 13:03:28.492700 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.492679    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.492788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.492720    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-metrics-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.492788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.492745    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-tmp\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.492788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.492761    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbb74\" (UniqueName: \"kubernetes.io/projected/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-kube-api-access-vbb74\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.492959 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.492939    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.493000 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.492987    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-manifests\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.515914 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.515841    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:28.552166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.552146    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 30 13:03:28.552239 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.552218    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tp27k\""
Oct 30 13:03:28.593850 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.593823    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-tmp\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.593948 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.593858    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbb74\" (UniqueName: \"kubernetes.io/projected/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-kube-api-access-vbb74\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.593948 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.593923    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.594057 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.593958    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-manifests\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.594057 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.594009    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.594157 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.594065    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-metrics-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.594377 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.594351    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-manifests\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.596110 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.596087    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-tmp\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.596345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.596322    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.596463 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.596446    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.596541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.596521    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-metrics-cert\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.648201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.648176    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbb74\" (UniqueName: \"kubernetes.io/projected/d640d2bd-3fc1-4d16-8aba-67e885e1a5ff-kube-api-access-vbb74\") pod \"opendatahub-operator-controller-manager-64d4d4766b-ckkkt\" (UID: \"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff\") " pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.733381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.733358    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:28.752333 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.752315    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:28.910801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:28.910770    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:29.196247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:29.196216    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:29.253030 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:29.253003    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"]
Oct 30 13:03:29.256537 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:03:29.256513    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd640d2bd_3fc1_4d16_8aba_67e885e1a5ff.slice/crio-ff408ac264620db11246d2f4750586c7c795103f1775661abfc18586789b847a WatchSource:0}: Error finding container ff408ac264620db11246d2f4750586c7c795103f1775661abfc18586789b847a: Status 404 returned error can't find the container with id ff408ac264620db11246d2f4750586c7c795103f1775661abfc18586789b847a
Oct 30 13:03:30.246350 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:30.246302    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt" event={"ID":"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff","Type":"ContainerStarted","Data":"ff408ac264620db11246d2f4750586c7c795103f1775661abfc18586789b847a"}
Oct 30 13:03:32.000825 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:03:32.000779    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:03:32.255906 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:32.255835    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt" event={"ID":"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff","Type":"ContainerStarted","Data":"131d6983e25905980fcae044d0afdfdd31a69045e05314bef6cde72ba13e13e2"}
Oct 30 13:03:33.260798 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:33.260768    5781 generic.go:356] "Generic (PLEG): container finished" podID="d640d2bd-3fc1-4d16-8aba-67e885e1a5ff" containerID="131d6983e25905980fcae044d0afdfdd31a69045e05314bef6cde72ba13e13e2" exitCode=0
Oct 30 13:03:33.261219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:33.260828    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt" event={"ID":"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff","Type":"ContainerDied","Data":"131d6983e25905980fcae044d0afdfdd31a69045e05314bef6cde72ba13e13e2"}
Oct 30 13:03:34.266198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:34.266166    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt" event={"ID":"d640d2bd-3fc1-4d16-8aba-67e885e1a5ff","Type":"ContainerStarted","Data":"8f55a76a7a4072c47a022142216da709fd37442efd087cc53a89b287cc486183"}
Oct 30 13:03:34.266580 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:34.266270    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:44.540509 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:44.540430    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:44.700131 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:44.700100    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:45.272563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:45.272538    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt"
Oct 30 13:03:45.394303 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:45.394233    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64d4d4766b-ckkkt" podStartSLOduration=13.806198485 podStartE2EDuration="17.394220786s" totalImagesPullingTime="3.102206298s" totalInitContainerRuntime="485.816003ms" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:34.365453451 +0000 UTC m=+351.662670992" watchObservedRunningTime="2026-10-30 13:03:45.394220786 +0000 UTC m=+362.691438328"
Oct 30 13:03:47.188588 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:47.188551    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:47.357909 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:47.357877    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:48.253649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:48.253612    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:48.462688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:48.462650    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:51.770372 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:51.770274    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:52.096614 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:52.096529    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:52.503283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:52.503250    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:54.803558 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:54.803520    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:59.623466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.623436    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:59.760374 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.760339    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"]
Oct 30 13:03:59.763987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.763967    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:03:59.779400 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.779381    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 13:03:59.790560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.790522    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 13:03:59.791856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.791831    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"]
Oct 30 13:03:59.805282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.805255    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 13:03:59.805785 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.805764    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:59.814226 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.814209    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 13:03:59.814338 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.814325    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wlkpx\""
Oct 30 13:03:59.860369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.860339    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55qpl\" (UniqueName: \"kubernetes.io/projected/f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a-kube-api-access-55qpl\") pod \"kube-auth-proxy-75fdd59bb7-hlxh8\" (UID: \"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:03:59.860480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.860404    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a-tls-certs\") pod \"kube-auth-proxy-75fdd59bb7-hlxh8\" (UID: \"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:03:59.860525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.860482    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a-tmp\") pod \"kube-auth-proxy-75fdd59bb7-hlxh8\" (UID: \"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:03:59.957602 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.957569    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:03:59.961658 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.961638    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55qpl\" (UniqueName: \"kubernetes.io/projected/f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a-kube-api-access-55qpl\") pod \"kube-auth-proxy-75fdd59bb7-hlxh8\" (UID: \"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:03:59.961726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.961690    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a-tls-certs\") pod \"kube-auth-proxy-75fdd59bb7-hlxh8\" (UID: \"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:03:59.961726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.961707    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a-tmp\") pod \"kube-auth-proxy-75fdd59bb7-hlxh8\" (UID: \"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:03:59.963835 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.963811    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a-tmp\") pod \"kube-auth-proxy-75fdd59bb7-hlxh8\" (UID: \"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:03:59.964142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.964123    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a-tls-certs\") pod \"kube-auth-proxy-75fdd59bb7-hlxh8\" (UID: \"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:03:59.987730 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:03:59.987711    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55qpl\" (UniqueName: \"kubernetes.io/projected/f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a-kube-api-access-55qpl\") pod \"kube-auth-proxy-75fdd59bb7-hlxh8\" (UID: \"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a\") " pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:04:00.076840 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:00.076806    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"
Oct 30 13:04:00.373266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:00.373196    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8"]
Oct 30 13:04:00.377951 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:04:00.377926    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0b8e3b0_3f54_4aa7_9461_2e50a8b0649a.slice/crio-11cd90c48f4e4f5710fa33c59408cc5b6f299385386c0a141db7c40898d1ab69 WatchSource:0}: Error finding container 11cd90c48f4e4f5710fa33c59408cc5b6f299385386c0a141db7c40898d1ab69: Status 404 returned error can't find the container with id 11cd90c48f4e4f5710fa33c59408cc5b6f299385386c0a141db7c40898d1ab69
Oct 30 13:04:01.377118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:01.377051    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8" event={"ID":"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a","Type":"ContainerStarted","Data":"11cd90c48f4e4f5710fa33c59408cc5b6f299385386c0a141db7c40898d1ab69"}
Oct 30 13:04:01.599180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:01.599139    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:01.751734 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:01.751705    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:01.999018 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:04:01.998953    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:04:03.061249 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:03.061166    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:03.387640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:03.387563    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8" event={"ID":"f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a","Type":"ContainerStarted","Data":"ab1fd063305a75f13b966e96bb5853b82340d63a1d04e6245d2a33c74859c6f6"}
Oct 30 13:04:03.428066 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:03.428012    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75fdd59bb7-hlxh8" podStartSLOduration=2.108291479 podStartE2EDuration="4.427998618s" totalImagesPullingTime="2.319707139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:03.427036306 +0000 UTC m=+380.724253848" watchObservedRunningTime="2026-10-30 13:04:03.427998618 +0000 UTC m=+380.725216160"
Oct 30 13:04:05.157551 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:05.157518    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:05.317275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:05.317245    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:06.630532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:06.630502    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:06.816184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:06.816146    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:06.999524 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:06.999489    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:14.902239 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:14.902203    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:15.108158 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:15.108124    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:15.325390 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:15.325351    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:31.995804 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:04:31.995765    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:04:33.057854 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:33.057822    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:34.267509 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:34.267476    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:55.465027 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:55.464990    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:04:59.891565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:04:59.891527    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:01.994439 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:05:01.994400    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:05:04.481483 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:04.481427    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:04.750795 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:04.750720    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:05.113007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.112927    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-jtrd7"]
Oct 30 13:05:05.116869 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.116841    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.149040 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.149002    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-jtrd7"]
Oct 30 13:05:05.167390 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.167354    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:05.206966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.206939    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 13:05:05.206966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.206949    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dlwct\""
Oct 30 13:05:05.206966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.206971    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 13:05:05.228512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.228477    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6546dc9b-551d-4dc0-988b-6a817d811f4a-metrics-certs\") pod \"odh-model-controller-f87fcf56c-jtrd7\" (UID: \"6546dc9b-551d-4dc0-988b-6a817d811f4a\") " pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.228645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.228515    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c78qk\" (UniqueName: \"kubernetes.io/projected/6546dc9b-551d-4dc0-988b-6a817d811f4a-kube-api-access-c78qk\") pod \"odh-model-controller-f87fcf56c-jtrd7\" (UID: \"6546dc9b-551d-4dc0-988b-6a817d811f4a\") " pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.228645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.228542    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6546dc9b-551d-4dc0-988b-6a817d811f4a-cert\") pod \"odh-model-controller-f87fcf56c-jtrd7\" (UID: \"6546dc9b-551d-4dc0-988b-6a817d811f4a\") " pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.329821 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.329787    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6546dc9b-551d-4dc0-988b-6a817d811f4a-metrics-certs\") pod \"odh-model-controller-f87fcf56c-jtrd7\" (UID: \"6546dc9b-551d-4dc0-988b-6a817d811f4a\") " pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.329990 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.329830    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c78qk\" (UniqueName: \"kubernetes.io/projected/6546dc9b-551d-4dc0-988b-6a817d811f4a-kube-api-access-c78qk\") pod \"odh-model-controller-f87fcf56c-jtrd7\" (UID: \"6546dc9b-551d-4dc0-988b-6a817d811f4a\") " pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.329990 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.329857    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6546dc9b-551d-4dc0-988b-6a817d811f4a-cert\") pod \"odh-model-controller-f87fcf56c-jtrd7\" (UID: \"6546dc9b-551d-4dc0-988b-6a817d811f4a\") " pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.332433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.332406    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6546dc9b-551d-4dc0-988b-6a817d811f4a-metrics-certs\") pod \"odh-model-controller-f87fcf56c-jtrd7\" (UID: \"6546dc9b-551d-4dc0-988b-6a817d811f4a\") " pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.332554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.332406    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6546dc9b-551d-4dc0-988b-6a817d811f4a-cert\") pod \"odh-model-controller-f87fcf56c-jtrd7\" (UID: \"6546dc9b-551d-4dc0-988b-6a817d811f4a\") " pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.358775 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.358743    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c78qk\" (UniqueName: \"kubernetes.io/projected/6546dc9b-551d-4dc0-988b-6a817d811f4a-kube-api-access-c78qk\") pod \"odh-model-controller-f87fcf56c-jtrd7\" (UID: \"6546dc9b-551d-4dc0-988b-6a817d811f4a\") " pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.429552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.429522    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:05.429975 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.429950    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:05.681783 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.681686    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:05.744853 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:05.744813    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-jtrd7"]
Oct 30 13:05:05.750070 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:05:05.750039    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6546dc9b_551d_4dc0_988b_6a817d811f4a.slice/crio-c4cb9403bbe4b08615539d4db1f49a4bfc2fac651503e6fde5d589b5d8aa26ee WatchSource:0}: Error finding container c4cb9403bbe4b08615539d4db1f49a4bfc2fac651503e6fde5d589b5d8aa26ee: Status 404 returned error can't find the container with id c4cb9403bbe4b08615539d4db1f49a4bfc2fac651503e6fde5d589b5d8aa26ee
Oct 30 13:05:06.647711 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:06.647669    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7" event={"ID":"6546dc9b-551d-4dc0-988b-6a817d811f4a","Type":"ContainerStarted","Data":"c4cb9403bbe4b08615539d4db1f49a4bfc2fac651503e6fde5d589b5d8aa26ee"}
Oct 30 13:05:07.618025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:07.617988    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:07.779528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:07.779499    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:07.931020 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:07.930988    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:08.178825 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:08.178791    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:08.657900 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:08.657821    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7" event={"ID":"6546dc9b-551d-4dc0-988b-6a817d811f4a","Type":"ContainerStarted","Data":"15fbf22d2bf046b57e98de493202772686495f1c246a8f8e2fb60f00efdb3d9e"}
Oct 30 13:05:08.658271 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:08.657956    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:08.699005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:08.698954    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7" podStartSLOduration=1.20770681 podStartE2EDuration="3.698941513s" totalImagesPullingTime="2.491234703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:08.697768006 +0000 UTC m=+445.994985548" watchObservedRunningTime="2026-10-30 13:05:08.698941513 +0000 UTC m=+445.996159055"
Oct 30 13:05:14.935864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:14.935830    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:19.664069 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:19.664039    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-f87fcf56c-jtrd7"
Oct 30 13:05:26.543244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:26.543151    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:28.402989 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:28.402952    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:05:31.994724 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:05:31.994686    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:05:55.089678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:05:55.089640    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:06:01.995061 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:06:01.995022    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:06:11.759740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:06:11.759697    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:06:11.930987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:06:11.930948    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:06:15.114827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:06:15.114781    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:06:31.996447 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:06:31.996337    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:06:55.173824 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:06:55.173736    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:01.995098 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:07:01.995064    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:07:04.873277 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:04.873245    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:09.888118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:09.888073    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:10.681101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:10.681065    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:13.892793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:13.892755    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:14.100894 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:14.100865    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:14.420339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:14.420307    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:14.647386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:14.647347    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:14.932922 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:14.932888    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:16.304490 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:16.304451    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:16.547080 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:16.547045    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:18.441910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:18.441872    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:19.020830 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:19.020797    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:20.668677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:20.668643    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:21.472029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:21.471992    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:23.319960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.319923    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"]
Oct 30 13:07:23.323703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.323678    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"
Oct 30 13:07:23.347566 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.347530    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 13:07:23.347737 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.347673    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 13:07:23.358954 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.358924    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 13:07:23.359115 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.359095    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dkbrg\""
Oct 30 13:07:23.360379 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.360358    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"]
Oct 30 13:07:23.379565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.379527    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjx45\" (UniqueName: \"kubernetes.io/projected/7566f692-4d21-4772-9e61-bbd0b220ece2-kube-api-access-cjx45\") pod \"dns-operator-controller-manager-65bfd79598-wsmjn\" (UID: \"7566f692-4d21-4772-9e61-bbd0b220ece2\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"
Oct 30 13:07:23.480283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.480249    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjx45\" (UniqueName: \"kubernetes.io/projected/7566f692-4d21-4772-9e61-bbd0b220ece2-kube-api-access-cjx45\") pod \"dns-operator-controller-manager-65bfd79598-wsmjn\" (UID: \"7566f692-4d21-4772-9e61-bbd0b220ece2\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"
Oct 30 13:07:23.520620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.520590    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjx45\" (UniqueName: \"kubernetes.io/projected/7566f692-4d21-4772-9e61-bbd0b220ece2-kube-api-access-cjx45\") pod \"dns-operator-controller-manager-65bfd79598-wsmjn\" (UID: \"7566f692-4d21-4772-9e61-bbd0b220ece2\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"
Oct 30 13:07:23.636799 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.636710    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"
Oct 30 13:07:23.920671 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.920644    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:23.961370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:23.961325    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"]
Oct 30 13:07:23.966480 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:07:23.966451    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7566f692_4d21_4772_9e61_bbd0b220ece2.slice/crio-2217e3c525ae62c576dcc94a6bdeff8b6fc001e306d469173ea82ddf6cfc9098 WatchSource:0}: Error finding container 2217e3c525ae62c576dcc94a6bdeff8b6fc001e306d469173ea82ddf6cfc9098: Status 404 returned error can't find the container with id 2217e3c525ae62c576dcc94a6bdeff8b6fc001e306d469173ea82ddf6cfc9098
Oct 30 13:07:24.132629 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:24.132595    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:24.247719 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:24.247626    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn" event={"ID":"7566f692-4d21-4772-9e61-bbd0b220ece2","Type":"ContainerStarted","Data":"2217e3c525ae62c576dcc94a6bdeff8b6fc001e306d469173ea82ddf6cfc9098"}
Oct 30 13:07:24.380149 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:24.380112    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:24.575953 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:24.575859    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:25.475445 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.475410    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"]
Oct 30 13:07:25.484038 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.484012    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.499451 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.499428    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 30 13:07:25.509854 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.509829    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-t5j5x\""
Oct 30 13:07:25.509854 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.509831    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 30 13:07:25.511872 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.511849    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"]
Oct 30 13:07:25.529942 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.529905    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:25.600225 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.600186    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4348bdb9-21fe-415c-b4c6-467611c09642-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-mv7mw\" (UID: \"4348bdb9-21fe-415c-b4c6-467611c09642\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.600225 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.600227    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4348bdb9-21fe-415c-b4c6-467611c09642-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-mv7mw\" (UID: \"4348bdb9-21fe-415c-b4c6-467611c09642\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.600465 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.600398    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlsg7\" (UniqueName: \"kubernetes.io/projected/4348bdb9-21fe-415c-b4c6-467611c09642-kube-api-access-mlsg7\") pod \"kuadrant-console-plugin-66c4fb575d-mv7mw\" (UID: \"4348bdb9-21fe-415c-b4c6-467611c09642\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.701475 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.701437    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4348bdb9-21fe-415c-b4c6-467611c09642-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-mv7mw\" (UID: \"4348bdb9-21fe-415c-b4c6-467611c09642\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.701635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.701481    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4348bdb9-21fe-415c-b4c6-467611c09642-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-mv7mw\" (UID: \"4348bdb9-21fe-415c-b4c6-467611c09642\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.701635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.701522    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlsg7\" (UniqueName: \"kubernetes.io/projected/4348bdb9-21fe-415c-b4c6-467611c09642-kube-api-access-mlsg7\") pod \"kuadrant-console-plugin-66c4fb575d-mv7mw\" (UID: \"4348bdb9-21fe-415c-b4c6-467611c09642\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.702253 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.702233    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4348bdb9-21fe-415c-b4c6-467611c09642-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-mv7mw\" (UID: \"4348bdb9-21fe-415c-b4c6-467611c09642\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.704065 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.704036    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4348bdb9-21fe-415c-b4c6-467611c09642-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-mv7mw\" (UID: \"4348bdb9-21fe-415c-b4c6-467611c09642\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.708899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.708869    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:25.729349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.729269    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlsg7\" (UniqueName: \"kubernetes.io/projected/4348bdb9-21fe-415c-b4c6-467611c09642-kube-api-access-mlsg7\") pod \"kuadrant-console-plugin-66c4fb575d-mv7mw\" (UID: \"4348bdb9-21fe-415c-b4c6-467611c09642\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:25.799417 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:25.799378    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"
Oct 30 13:07:26.108022 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:26.107905    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw"]
Oct 30 13:07:26.113583 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:07:26.113555    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4348bdb9_21fe_415c_b4c6_467611c09642.slice/crio-d3b2644c1f0e370dc8fdcb433c1d81c5935b3461adb08bc9bca2a7387a53a46e WatchSource:0}: Error finding container d3b2644c1f0e370dc8fdcb433c1d81c5935b3461adb08bc9bca2a7387a53a46e: Status 404 returned error can't find the container with id d3b2644c1f0e370dc8fdcb433c1d81c5935b3461adb08bc9bca2a7387a53a46e
Oct 30 13:07:26.258456 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:26.258417    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw" event={"ID":"4348bdb9-21fe-415c-b4c6-467611c09642","Type":"ContainerStarted","Data":"d3b2644c1f0e370dc8fdcb433c1d81c5935b3461adb08bc9bca2a7387a53a46e"}
Oct 30 13:07:27.268777 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:27.267758    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn" event={"ID":"7566f692-4d21-4772-9e61-bbd0b220ece2","Type":"ContainerStarted","Data":"46da83822f7975d176a0c944b80df640b17dc8d338602ea74844bcad9fbc529a"}
Oct 30 13:07:27.268777 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:27.268710    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"
Oct 30 13:07:27.319324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:27.318880    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn" podStartSLOduration=1.8808241369999998 podStartE2EDuration="4.318860244s" totalImagesPullingTime="2.438036107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:07:27.315595233 +0000 UTC m=+584.612812775" watchObservedRunningTime="2026-10-30 13:07:27.318860244 +0000 UTC m=+584.616077788"
Oct 30 13:07:29.782018 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:29.781977    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:29.949471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:29.949434    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:30.104955 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:30.104876    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:30.314004 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:30.313972    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:31.289041 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:31.289000    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw" event={"ID":"4348bdb9-21fe-415c-b4c6-467611c09642","Type":"ContainerStarted","Data":"8a7b72a0d82ccc07f485a6f16bed78f839c99d77c1fb4de85b05fa5221116972"}
Oct 30 13:07:31.333145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:31.333091    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-mv7mw" podStartSLOduration=1.9660297679999998 podStartE2EDuration="6.333072366s" totalImagesPullingTime="4.367042598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:07:31.331112296 +0000 UTC m=+588.628329840" watchObservedRunningTime="2026-10-30 13:07:31.333072366 +0000 UTC m=+588.630289909"
Oct 30 13:07:31.995401 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:07:31.995366    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:07:36.390885 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.390849    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:36.569676 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.569630    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:36.588126 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.588096    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"]
Oct 30 13:07:36.592769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.592746    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:36.619069 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.619032    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6szdh\""
Oct 30 13:07:36.625656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.625624    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"]
Oct 30 13:07:36.707749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.707698    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfcq2\" (UniqueName: \"kubernetes.io/projected/fdb268a9-1f93-41bc-b250-49a843431d92-kube-api-access-xfcq2\") pod \"kuadrant-operator-controller-manager-956956bc4-fsq48\" (UID: \"fdb268a9-1f93-41bc-b250-49a843431d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:36.707937 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.707770    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdb268a9-1f93-41bc-b250-49a843431d92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-fsq48\" (UID: \"fdb268a9-1f93-41bc-b250-49a843431d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:36.808785 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.808723    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdb268a9-1f93-41bc-b250-49a843431d92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-fsq48\" (UID: \"fdb268a9-1f93-41bc-b250-49a843431d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:36.808978 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.808841    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfcq2\" (UniqueName: \"kubernetes.io/projected/fdb268a9-1f93-41bc-b250-49a843431d92-kube-api-access-xfcq2\") pod \"kuadrant-operator-controller-manager-956956bc4-fsq48\" (UID: \"fdb268a9-1f93-41bc-b250-49a843431d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:36.809153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.809132    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdb268a9-1f93-41bc-b250-49a843431d92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-fsq48\" (UID: \"fdb268a9-1f93-41bc-b250-49a843431d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:36.829282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.829248    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"]
Oct 30 13:07:36.829608 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:07:36.829581    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xfcq2], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48" podUID="fdb268a9-1f93-41bc-b250-49a843431d92"
Oct 30 13:07:36.839198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:36.839169    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfcq2\" (UniqueName: \"kubernetes.io/projected/fdb268a9-1f93-41bc-b250-49a843431d92-kube-api-access-xfcq2\") pod \"kuadrant-operator-controller-manager-956956bc4-fsq48\" (UID: \"fdb268a9-1f93-41bc-b250-49a843431d92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:37.313318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:37.313270    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:37.320891 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:37.320865    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:37.413096 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:37.413053    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fdb268a9-1f93-41bc-b250-49a843431d92-kube-api-access-xfcq2\" (UniqueName: \"kubernetes.io/projected/fdb268a9-1f93-41bc-b250-49a843431d92-kube-api-access-xfcq2\") pod \"fdb268a9-1f93-41bc-b250-49a843431d92\" (UID: \"fdb268a9-1f93-41bc-b250-49a843431d92\") "
Oct 30 13:07:37.413581 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:37.413118    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fdb268a9-1f93-41bc-b250-49a843431d92-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdb268a9-1f93-41bc-b250-49a843431d92-extensions-socket-volume\") pod \"fdb268a9-1f93-41bc-b250-49a843431d92\" (UID: \"fdb268a9-1f93-41bc-b250-49a843431d92\") "
Oct 30 13:07:37.413581 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:37.413395    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fdb268a9-1f93-41bc-b250-49a843431d92-extensions-socket-volume" pod "fdb268a9-1f93-41bc-b250-49a843431d92" (UID: "fdb268a9-1f93-41bc-b250-49a843431d92"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:37.415267 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:37.415244    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdb268a9-1f93-41bc-b250-49a843431d92-kube-api-access-xfcq2" pod "fdb268a9-1f93-41bc-b250-49a843431d92" (UID: "fdb268a9-1f93-41bc-b250-49a843431d92"). InnerVolumeSpecName "kube-api-access-xfcq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:37.514669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:37.514632    5781 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdb268a9-1f93-41bc-b250-49a843431d92-extensions-socket-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:07:37.514669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:37.514662    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfcq2\" (UniqueName: \"kubernetes.io/projected/fdb268a9-1f93-41bc-b250-49a843431d92-kube-api-access-xfcq2\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:07:38.125575 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:38.125539    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"]
Oct 30 13:07:38.137322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:38.137276    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"]
Oct 30 13:07:38.317691 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:38.317659    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:38.331317 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:07:38.331272    5781 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-956956bc4-fsq48\" is forbidden: User \"system:node:ip-10-0-130-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-180.ec2.internal' and this object" podUID="fdb268a9-1f93-41bc-b250-49a843431d92" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:38.344138 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:07:38.344102    5781 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-956956bc4-fsq48\" is forbidden: User \"system:node:ip-10-0-130-180.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-180.ec2.internal' and this object" podUID="fdb268a9-1f93-41bc-b250-49a843431d92" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-fsq48"
Oct 30 13:07:38.839673 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:38.839637    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fdb268a9-1f93-41bc-b250-49a843431d92" path="/var/lib/kubelet/pods/fdb268a9-1f93-41bc-b250-49a843431d92/volumes"
Oct 30 13:07:38.865909 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:38.865871    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:39.167952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:39.167852    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:39.280227 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:39.280190    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-wsmjn"
Oct 30 13:07:42.895072 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:42.895043    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:07:42.895565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:42.895145    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:07:44.031244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:44.031204    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:44.227995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:44.227964    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:48.846545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:48.846508    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:49.043386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:49.043349    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:49.238259 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:49.238225    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:51.099531 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:51.099470    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:55.432708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:55.432672    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:59.096848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:59.096815    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:07:59.299478 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:07:59.299442    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:01.995828 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:08:01.995783    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:08:02.079309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:02.079243    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:02.246273 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:02.246186    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:17.852904 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:17.852839    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:18.014256 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:18.014227    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:20.582510 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.582474    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-pvppc"]
Oct 30 13:08:20.585920 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.585899    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:20.602867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.602846    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 13:08:20.613309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.613259    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-pvppc"]
Oct 30 13:08:20.663317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.663250    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:20.690758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.690722    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59dnc\" (UniqueName: \"kubernetes.io/projected/495051a7-b506-4a0f-b7b7-0d59f82aa96c-kube-api-access-59dnc\") pod \"limitador-limitador-74dfcfb894-pvppc\" (UID: \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:20.690883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.690859    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/495051a7-b506-4a0f-b7b7-0d59f82aa96c-config-file\") pod \"limitador-limitador-74dfcfb894-pvppc\" (UID: \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:20.710050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.710017    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-pvppc"]
Oct 30 13:08:20.792157 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.792134    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/495051a7-b506-4a0f-b7b7-0d59f82aa96c-config-file\") pod \"limitador-limitador-74dfcfb894-pvppc\" (UID: \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:20.792318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.792202    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59dnc\" (UniqueName: \"kubernetes.io/projected/495051a7-b506-4a0f-b7b7-0d59f82aa96c-kube-api-access-59dnc\") pod \"limitador-limitador-74dfcfb894-pvppc\" (UID: \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:20.792776 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.792756    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/495051a7-b506-4a0f-b7b7-0d59f82aa96c-config-file\") pod \"limitador-limitador-74dfcfb894-pvppc\" (UID: \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:20.820757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.820734    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59dnc\" (UniqueName: \"kubernetes.io/projected/495051a7-b506-4a0f-b7b7-0d59f82aa96c-kube-api-access-59dnc\") pod \"limitador-limitador-74dfcfb894-pvppc\" (UID: \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:20.849549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.849462    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:20.898890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:20.898868    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:21.194607 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:21.194578    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-pvppc"]
Oct 30 13:08:21.199696 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:08:21.199661    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod495051a7_b506_4a0f_b7b7_0d59f82aa96c.slice/crio-604d7749a31d25be8db0f9bd9832cad2abbdf915565395bc89cee4b1b8262c7e WatchSource:0}: Error finding container 604d7749a31d25be8db0f9bd9832cad2abbdf915565395bc89cee4b1b8262c7e: Status 404 returned error can't find the container with id 604d7749a31d25be8db0f9bd9832cad2abbdf915565395bc89cee4b1b8262c7e
Oct 30 13:08:21.202044 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:21.202025    5781 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:08:21.508330 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:21.508241    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc" event={"ID":"495051a7-b506-4a0f-b7b7-0d59f82aa96c","Type":"ContainerStarted","Data":"604d7749a31d25be8db0f9bd9832cad2abbdf915565395bc89cee4b1b8262c7e"}
Oct 30 13:08:21.622146 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:21.622073    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:21.803579 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:21.803499    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:22.191411 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:22.191374    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bcgqq"]
Oct 30 13:08:22.195702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:22.195675    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq"
Oct 30 13:08:22.211688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:22.211668    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m59kx\""
Oct 30 13:08:22.222190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:22.222169    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bcgqq"]
Oct 30 13:08:22.303949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:22.303914    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5clkm\" (UniqueName: \"kubernetes.io/projected/4dacb3cc-221b-4f92-8204-2dc0778b44e6-kube-api-access-5clkm\") pod \"authorino-56c6c9ddd5-bcgqq\" (UID: \"4dacb3cc-221b-4f92-8204-2dc0778b44e6\") " pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq"
Oct 30 13:08:22.405488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:22.405456    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5clkm\" (UniqueName: \"kubernetes.io/projected/4dacb3cc-221b-4f92-8204-2dc0778b44e6-kube-api-access-5clkm\") pod \"authorino-56c6c9ddd5-bcgqq\" (UID: \"4dacb3cc-221b-4f92-8204-2dc0778b44e6\") " pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq"
Oct 30 13:08:22.428859 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:22.428825    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5clkm\" (UniqueName: \"kubernetes.io/projected/4dacb3cc-221b-4f92-8204-2dc0778b44e6-kube-api-access-5clkm\") pod \"authorino-56c6c9ddd5-bcgqq\" (UID: \"4dacb3cc-221b-4f92-8204-2dc0778b44e6\") " pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq"
Oct 30 13:08:22.509208 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:22.509134    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq"
Oct 30 13:08:22.811059 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:22.810982    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bcgqq"]
Oct 30 13:08:22.818487 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:08:22.818443    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dacb3cc_221b_4f92_8204_2dc0778b44e6.slice/crio-bfdb16dd48fd7ef41284e218a5b816a155ae812cff6e8102bebe915068f2cb35 WatchSource:0}: Error finding container bfdb16dd48fd7ef41284e218a5b816a155ae812cff6e8102bebe915068f2cb35: Status 404 returned error can't find the container with id bfdb16dd48fd7ef41284e218a5b816a155ae812cff6e8102bebe915068f2cb35
Oct 30 13:08:23.519090 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:23.519039    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq" event={"ID":"4dacb3cc-221b-4f92-8204-2dc0778b44e6","Type":"ContainerStarted","Data":"bfdb16dd48fd7ef41284e218a5b816a155ae812cff6e8102bebe915068f2cb35"}
Oct 30 13:08:23.520720 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:23.520695    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc" event={"ID":"495051a7-b506-4a0f-b7b7-0d59f82aa96c","Type":"ContainerStarted","Data":"2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f"}
Oct 30 13:08:23.520858 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:23.520831    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:23.562180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:23.562134    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc" podStartSLOduration=2.177865911 podStartE2EDuration="3.562121228s" totalImagesPullingTime="1.384255317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:08:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:08:23.560553814 +0000 UTC m=+640.857771356" watchObservedRunningTime="2026-10-30 13:08:23.562121228 +0000 UTC m=+640.859338770"
Oct 30 13:08:24.528203 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:24.528163    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq" event={"ID":"4dacb3cc-221b-4f92-8204-2dc0778b44e6","Type":"ContainerStarted","Data":"76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7"}
Oct 30 13:08:24.583581 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:24.583522    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq" podStartSLOduration=1.042267962 podStartE2EDuration="2.583506975s" totalImagesPullingTime="1.541239013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:08:24.581644841 +0000 UTC m=+641.878862384" watchObservedRunningTime="2026-10-30 13:08:24.583506975 +0000 UTC m=+641.880724519"
Oct 30 13:08:25.057660 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:25.057604    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:31.995232 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:08:31.995188    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:08:34.529664 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:34.529636    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:35.406489 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:35.406449    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-pvppc"]
Oct 30 13:08:35.406758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:35.406662    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc" podUID="495051a7-b506-4a0f-b7b7-0d59f82aa96c" containerName="limitador" containerID="cri-o://2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f" gracePeriod=30
Oct 30 13:08:35.962274 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:35.962250    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:36.130282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.130196    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/495051a7-b506-4a0f-b7b7-0d59f82aa96c-kube-api-access-59dnc\" (UniqueName: \"kubernetes.io/projected/495051a7-b506-4a0f-b7b7-0d59f82aa96c-kube-api-access-59dnc\") pod \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\" (UID: \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\") "
Oct 30 13:08:36.130282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.130297    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/495051a7-b506-4a0f-b7b7-0d59f82aa96c-config-file\" (UniqueName: \"kubernetes.io/configmap/495051a7-b506-4a0f-b7b7-0d59f82aa96c-config-file\") pod \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\" (UID: \"495051a7-b506-4a0f-b7b7-0d59f82aa96c\") "
Oct 30 13:08:36.130709 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.130684    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/495051a7-b506-4a0f-b7b7-0d59f82aa96c-config-file" pod "495051a7-b506-4a0f-b7b7-0d59f82aa96c" (UID: "495051a7-b506-4a0f-b7b7-0d59f82aa96c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:08:36.132566 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.132544    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/495051a7-b506-4a0f-b7b7-0d59f82aa96c-kube-api-access-59dnc" pod "495051a7-b506-4a0f-b7b7-0d59f82aa96c" (UID: "495051a7-b506-4a0f-b7b7-0d59f82aa96c"). InnerVolumeSpecName "kube-api-access-59dnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:08:36.231373 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.231331    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-59dnc\" (UniqueName: \"kubernetes.io/projected/495051a7-b506-4a0f-b7b7-0d59f82aa96c-kube-api-access-59dnc\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:08:36.231373 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.231366    5781 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/495051a7-b506-4a0f-b7b7-0d59f82aa96c-config-file\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:08:36.583471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.583434    5781 generic.go:356] "Generic (PLEG): container finished" podID="495051a7-b506-4a0f-b7b7-0d59f82aa96c" containerID="2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f" exitCode=0
Oct 30 13:08:36.583636 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.583499    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc"
Oct 30 13:08:36.583636 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.583505    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc" event={"ID":"495051a7-b506-4a0f-b7b7-0d59f82aa96c","Type":"ContainerDied","Data":"2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f"}
Oct 30 13:08:36.583636 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.583584    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-pvppc" event={"ID":"495051a7-b506-4a0f-b7b7-0d59f82aa96c","Type":"ContainerDied","Data":"604d7749a31d25be8db0f9bd9832cad2abbdf915565395bc89cee4b1b8262c7e"}
Oct 30 13:08:36.583636 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.583606    5781 scope.go:122] "RemoveContainer" containerID="2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f"
Oct 30 13:08:36.595887 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.595865    5781 scope.go:122] "RemoveContainer" containerID="2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f"
Oct 30 13:08:36.596233 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:08:36.596210    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f\": container with ID starting with 2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f not found: ID does not exist" containerID="2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f"
Oct 30 13:08:36.596274 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.596244    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f"} err="failed to get container status \"2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f\": rpc error: code = NotFound desc = could not find container \"2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f\": container with ID starting with 2cc5868ea58f8c8ea4cca0a9984dfecac2b293b494490911ab78acc44d3c665f not found: ID does not exist"
Oct 30 13:08:36.631757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.631721    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-pvppc"]
Oct 30 13:08:36.643223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.643192    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-pvppc"]
Oct 30 13:08:36.839871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:36.839797    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="495051a7-b506-4a0f-b7b7-0d59f82aa96c" path="/var/lib/kubelet/pods/495051a7-b506-4a0f-b7b7-0d59f82aa96c/volumes"
Oct 30 13:08:51.510664 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:51.510629    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:51.695072 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:51.695039    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:52.191669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.191636    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"]
Oct 30 13:08:52.192118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.192075    5781 cpu_manager.go:414] "containerMap: removing container" podUID="495051a7-b506-4a0f-b7b7-0d59f82aa96c" containerName="limitador"
Oct 30 13:08:52.192118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.192095    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="495051a7-b506-4a0f-b7b7-0d59f82aa96c" containerName="limitador"
Oct 30 13:08:52.192394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.192237    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="495051a7-b506-4a0f-b7b7-0d59f82aa96c" containerName="limitador"
Oct 30 13:08:52.196773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.196753    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:08:52.216065 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.216029    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-59v7x\""
Oct 30 13:08:52.227640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.227612    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 13:08:52.227823 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.227650    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 13:08:52.228930 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.228903    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"]
Oct 30 13:08:52.259595 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.259566    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c522dc11-5cd0-4b4a-8003-1400d800d815-data\") pod \"postgres-7cdbf6dc44-c5ntv\" (UID: \"c522dc11-5cd0-4b4a-8003-1400d800d815\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:08:52.259776 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.259620    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6tzd\" (UniqueName: \"kubernetes.io/projected/c522dc11-5cd0-4b4a-8003-1400d800d815-kube-api-access-z6tzd\") pod \"postgres-7cdbf6dc44-c5ntv\" (UID: \"c522dc11-5cd0-4b4a-8003-1400d800d815\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:08:52.321677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.321644    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 13:08:52.360249 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.360212    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c522dc11-5cd0-4b4a-8003-1400d800d815-data\") pod \"postgres-7cdbf6dc44-c5ntv\" (UID: \"c522dc11-5cd0-4b4a-8003-1400d800d815\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:08:52.360396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.360310    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6tzd\" (UniqueName: \"kubernetes.io/projected/c522dc11-5cd0-4b4a-8003-1400d800d815-kube-api-access-z6tzd\") pod \"postgres-7cdbf6dc44-c5ntv\" (UID: \"c522dc11-5cd0-4b4a-8003-1400d800d815\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:08:52.360630 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.360608    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c522dc11-5cd0-4b4a-8003-1400d800d815-data\") pod \"postgres-7cdbf6dc44-c5ntv\" (UID: \"c522dc11-5cd0-4b4a-8003-1400d800d815\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:08:52.391259 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.391225    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6tzd\" (UniqueName: \"kubernetes.io/projected/c522dc11-5cd0-4b4a-8003-1400d800d815-kube-api-access-z6tzd\") pod \"postgres-7cdbf6dc44-c5ntv\" (UID: \"c522dc11-5cd0-4b4a-8003-1400d800d815\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:08:52.447346 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.447244    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:52.509552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.509514    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:08:52.807767 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:52.807686    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"]
Oct 30 13:08:52.812759 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:08:52.812730    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc522dc11_5cd0_4b4a_8003_1400d800d815.slice/crio-065b75b8fe9e8ed37975150dc228b9498b1efbb57979f8fbcf1024e355f5cd4e WatchSource:0}: Error finding container 065b75b8fe9e8ed37975150dc228b9498b1efbb57979f8fbcf1024e355f5cd4e: Status 404 returned error can't find the container with id 065b75b8fe9e8ed37975150dc228b9498b1efbb57979f8fbcf1024e355f5cd4e
Oct 30 13:08:53.662484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:53.662443    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv" event={"ID":"c522dc11-5cd0-4b4a-8003-1400d800d815","Type":"ContainerStarted","Data":"065b75b8fe9e8ed37975150dc228b9498b1efbb57979f8fbcf1024e355f5cd4e"}
Oct 30 13:08:55.205525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:55.205479    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:08:58.687385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:58.687349    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv" event={"ID":"c522dc11-5cd0-4b4a-8003-1400d800d815","Type":"ContainerStarted","Data":"d1c06a8b652845612c88ec2400f6823de225d58212be74d8602bd98ff5dc7fe1"}
Oct 30 13:08:58.687809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:58.687437    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:08:58.726980 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:08:58.726917    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv" podStartSLOduration=1.756552148 podStartE2EDuration="6.726899706s" totalImagesPullingTime="4.970347558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:08:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:08:58.725710847 +0000 UTC m=+676.022928389" watchObservedRunningTime="2026-10-30 13:08:58.726899706 +0000 UTC m=+676.024117250"
Oct 30 13:09:01.557981 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:01.557948    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:01.707512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:01.707478    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:01.995522 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:09:01.995481    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:09:04.721638 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:04.721607    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-c5ntv"
Oct 30 13:09:05.220873 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.220836    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:05.500532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.500450    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-67c5d"]
Oct 30 13:09:05.504348 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.504325    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-67c5d"
Oct 30 13:09:05.527065 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.527035    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-67c5d"]
Oct 30 13:09:05.578817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.578781    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:05.601644 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.601602    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx7cg\" (UniqueName: \"kubernetes.io/projected/888d9686-a540-45de-b891-0eefa957638b-kube-api-access-cx7cg\") pod \"authorino-6f6944bcdd-67c5d\" (UID: \"888d9686-a540-45de-b891-0eefa957638b\") " pod="kuadrant-system/authorino-6f6944bcdd-67c5d"
Oct 30 13:09:05.697474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.697439    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-67c5d"]
Oct 30 13:09:05.697682 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:09:05.697663    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-cx7cg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f6944bcdd-67c5d" podUID="888d9686-a540-45de-b891-0eefa957638b"
Oct 30 13:09:05.702666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.702633    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx7cg\" (UniqueName: \"kubernetes.io/projected/888d9686-a540-45de-b891-0eefa957638b-kube-api-access-cx7cg\") pod \"authorino-6f6944bcdd-67c5d\" (UID: \"888d9686-a540-45de-b891-0eefa957638b\") " pod="kuadrant-system/authorino-6f6944bcdd-67c5d"
Oct 30 13:09:05.719103 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.719078    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-67c5d"
Oct 30 13:09:05.726121 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.726098    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-67c5d"
Oct 30 13:09:05.729069 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.729042    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx7cg\" (UniqueName: \"kubernetes.io/projected/888d9686-a540-45de-b891-0eefa957638b-kube-api-access-cx7cg\") pod \"authorino-6f6944bcdd-67c5d\" (UID: \"888d9686-a540-45de-b891-0eefa957638b\") " pod="kuadrant-system/authorino-6f6944bcdd-67c5d"
Oct 30 13:09:05.731935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.731916    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-jnst9"]
Oct 30 13:09:05.736328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.736309    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:05.750744 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.750695    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 13:09:05.758389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.758368    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-jnst9"]
Oct 30 13:09:05.803811 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.803769    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/888d9686-a540-45de-b891-0eefa957638b-kube-api-access-cx7cg\" (UniqueName: \"kubernetes.io/projected/888d9686-a540-45de-b891-0eefa957638b-kube-api-access-cx7cg\") pod \"888d9686-a540-45de-b891-0eefa957638b\" (UID: \"888d9686-a540-45de-b891-0eefa957638b\") "
Oct 30 13:09:05.804020 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.804003    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd68f\" (UniqueName: \"kubernetes.io/projected/0cb4c0d6-ba04-4de6-b750-d22acf95e815-kube-api-access-pd68f\") pod \"authorino-6889cbd8c4-jnst9\" (UID: \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\") " pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:05.804091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.804078    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0cb4c0d6-ba04-4de6-b750-d22acf95e815-tls-cert\") pod \"authorino-6889cbd8c4-jnst9\" (UID: \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\") " pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:05.805856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.805832    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/888d9686-a540-45de-b891-0eefa957638b-kube-api-access-cx7cg" pod "888d9686-a540-45de-b891-0eefa957638b" (UID: "888d9686-a540-45de-b891-0eefa957638b"). InnerVolumeSpecName "kube-api-access-cx7cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:05.845094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.845055    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-jnst9"]
Oct 30 13:09:05.845325 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:09:05.845301    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pd68f tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6889cbd8c4-jnst9" podUID="0cb4c0d6-ba04-4de6-b750-d22acf95e815"
Oct 30 13:09:05.905012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.904975    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0cb4c0d6-ba04-4de6-b750-d22acf95e815-tls-cert\") pod \"authorino-6889cbd8c4-jnst9\" (UID: \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\") " pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:05.905214 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.905045    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd68f\" (UniqueName: \"kubernetes.io/projected/0cb4c0d6-ba04-4de6-b750-d22acf95e815-kube-api-access-pd68f\") pod \"authorino-6889cbd8c4-jnst9\" (UID: \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\") " pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:05.905214 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.905098    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx7cg\" (UniqueName: \"kubernetes.io/projected/888d9686-a540-45de-b891-0eefa957638b-kube-api-access-cx7cg\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:09:05.907621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.907591    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0cb4c0d6-ba04-4de6-b750-d22acf95e815-tls-cert\") pod \"authorino-6889cbd8c4-jnst9\" (UID: \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\") " pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:05.928783 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:05.928749    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd68f\" (UniqueName: \"kubernetes.io/projected/0cb4c0d6-ba04-4de6-b750-d22acf95e815-kube-api-access-pd68f\") pod \"authorino-6889cbd8c4-jnst9\" (UID: \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\") " pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:06.723155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.723124    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-67c5d"
Oct 30 13:09:06.723394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.723124    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:06.730951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.730930    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:06.811165 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.811127    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0cb4c0d6-ba04-4de6-b750-d22acf95e815-tls-cert\" (UniqueName: \"kubernetes.io/secret/0cb4c0d6-ba04-4de6-b750-d22acf95e815-tls-cert\") pod \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\" (UID: \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\") "
Oct 30 13:09:06.811415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.811201    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cb4c0d6-ba04-4de6-b750-d22acf95e815-kube-api-access-pd68f\" (UniqueName: \"kubernetes.io/projected/0cb4c0d6-ba04-4de6-b750-d22acf95e815-kube-api-access-pd68f\") pod \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\" (UID: \"0cb4c0d6-ba04-4de6-b750-d22acf95e815\") "
Oct 30 13:09:06.813651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.813619    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cb4c0d6-ba04-4de6-b750-d22acf95e815-tls-cert" pod "0cb4c0d6-ba04-4de6-b750-d22acf95e815" (UID: "0cb4c0d6-ba04-4de6-b750-d22acf95e815"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:09:06.813651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.813632    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb4c0d6-ba04-4de6-b750-d22acf95e815-kube-api-access-pd68f" pod "0cb4c0d6-ba04-4de6-b750-d22acf95e815" (UID: "0cb4c0d6-ba04-4de6-b750-d22acf95e815"). InnerVolumeSpecName "kube-api-access-pd68f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:06.832679 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.832642    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-67c5d"]
Oct 30 13:09:06.846840 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.846806    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-67c5d"]
Oct 30 13:09:06.912032 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.911993    5781 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0cb4c0d6-ba04-4de6-b750-d22acf95e815-tls-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:09:06.912032 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:06.912023    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd68f\" (UniqueName: \"kubernetes.io/projected/0cb4c0d6-ba04-4de6-b750-d22acf95e815-kube-api-access-pd68f\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:09:07.622166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:07.622131    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bcgqq"]
Oct 30 13:09:07.622358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:07.622339    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq" podUID="4dacb3cc-221b-4f92-8204-2dc0778b44e6" containerName="authorino" containerID="cri-o://76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7" gracePeriod=30
Oct 30 13:09:07.727474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:07.727446    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6889cbd8c4-jnst9"
Oct 30 13:09:07.802647 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:07.802611    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-jnst9"]
Oct 30 13:09:07.805875 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:07.805841    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6889cbd8c4-jnst9"]
Oct 30 13:09:07.865222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:07.865195    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq"
Oct 30 13:09:07.922084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:07.922060    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dacb3cc-221b-4f92-8204-2dc0778b44e6-kube-api-access-5clkm\" (UniqueName: \"kubernetes.io/projected/4dacb3cc-221b-4f92-8204-2dc0778b44e6-kube-api-access-5clkm\") pod \"4dacb3cc-221b-4f92-8204-2dc0778b44e6\" (UID: \"4dacb3cc-221b-4f92-8204-2dc0778b44e6\") "
Oct 30 13:09:07.924117 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:07.924084    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dacb3cc-221b-4f92-8204-2dc0778b44e6-kube-api-access-5clkm" pod "4dacb3cc-221b-4f92-8204-2dc0778b44e6" (UID: "4dacb3cc-221b-4f92-8204-2dc0778b44e6"). InnerVolumeSpecName "kube-api-access-5clkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:08.022880 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.022839    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5clkm\" (UniqueName: \"kubernetes.io/projected/4dacb3cc-221b-4f92-8204-2dc0778b44e6-kube-api-access-5clkm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:09:08.732605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.732572    5781 generic.go:356] "Generic (PLEG): container finished" podID="4dacb3cc-221b-4f92-8204-2dc0778b44e6" containerID="76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7" exitCode=0
Oct 30 13:09:08.732793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.732617    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq"
Oct 30 13:09:08.732793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.732651    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq" event={"ID":"4dacb3cc-221b-4f92-8204-2dc0778b44e6","Type":"ContainerDied","Data":"76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7"}
Oct 30 13:09:08.732793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.732681    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-bcgqq" event={"ID":"4dacb3cc-221b-4f92-8204-2dc0778b44e6","Type":"ContainerDied","Data":"bfdb16dd48fd7ef41284e218a5b816a155ae812cff6e8102bebe915068f2cb35"}
Oct 30 13:09:08.732793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.732699    5781 scope.go:122] "RemoveContainer" containerID="76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7"
Oct 30 13:09:08.741544 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.741518    5781 scope.go:122] "RemoveContainer" containerID="76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7"
Oct 30 13:09:08.741819 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:09:08.741801    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7\": container with ID starting with 76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7 not found: ID does not exist" containerID="76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7"
Oct 30 13:09:08.741898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.741829    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7"} err="failed to get container status \"76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7\": rpc error: code = NotFound desc = could not find container \"76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7\": container with ID starting with 76395d34d39ae1c8d554029f68946ce5302edf640e0b1c32b9f440fcf03c1ad7 not found: ID does not exist"
Oct 30 13:09:08.780362 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.780331    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bcgqq"]
Oct 30 13:09:08.784664 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.784637    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-bcgqq"]
Oct 30 13:09:08.839448 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.839417    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0cb4c0d6-ba04-4de6-b750-d22acf95e815" path="/var/lib/kubelet/pods/0cb4c0d6-ba04-4de6-b750-d22acf95e815/volumes"
Oct 30 13:09:08.839852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.839638    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4dacb3cc-221b-4f92-8204-2dc0778b44e6" path="/var/lib/kubelet/pods/4dacb3cc-221b-4f92-8204-2dc0778b44e6/volumes"
Oct 30 13:09:08.839924 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:08.839912    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="888d9686-a540-45de-b891-0eefa957638b" path="/var/lib/kubelet/pods/888d9686-a540-45de-b891-0eefa957638b/volumes"
Oct 30 13:09:15.050685 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.050640    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:15.095619 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.095586    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577f596567-6xv5m"]
Oct 30 13:09:15.096008 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.095992    5781 cpu_manager.go:414] "containerMap: removing container" podUID="4dacb3cc-221b-4f92-8204-2dc0778b44e6" containerName="authorino"
Oct 30 13:09:15.096008 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.096008    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dacb3cc-221b-4f92-8204-2dc0778b44e6" containerName="authorino"
Oct 30 13:09:15.096200 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.096109    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dacb3cc-221b-4f92-8204-2dc0778b44e6" containerName="authorino"
Oct 30 13:09:15.100396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.100379    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.117869 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.117838    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 13:09:15.117869 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.117851    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 13:09:15.129035 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.129013    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 13:09:15.129178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.129163    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kvhzq\""
Oct 30 13:09:15.130697 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.130670    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-6xv5m"]
Oct 30 13:09:15.189396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.189358    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.189396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.189392    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjlw7\" (UniqueName: \"kubernetes.io/projected/d033bacb-931e-4d59-bc4e-1b1685b379e2-kube-api-access-mjlw7\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.189751 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.189604    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-metrics-certs\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.290132 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.290092    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-metrics-certs\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.290342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.290141    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.290342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.290158    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjlw7\" (UniqueName: \"kubernetes.io/projected/d033bacb-931e-4d59-bc4e-1b1685b379e2-kube-api-access-mjlw7\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.290342 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:09:15.290276    5781 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Oct 30 13:09:15.290453 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:09:15.290373    5781 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert podName:d033bacb-931e-4d59-bc4e-1b1685b379e2 nodeName:}" failed. No retries permitted until 2026-10-30 13:09:15.7903555 +0000 UTC m=+693.087573020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert") pod "maas-controller-577f596567-6xv5m" (UID: "d033bacb-931e-4d59-bc4e-1b1685b379e2") : secret "maas-controller-webhook-cert" not found
Oct 30 13:09:15.292629 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.292608    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-metrics-certs\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.319466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.319388    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjlw7\" (UniqueName: \"kubernetes.io/projected/d033bacb-931e-4d59-bc4e-1b1685b379e2-kube-api-access-mjlw7\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.328834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.328777    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:15.499145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.499107    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:15.513336 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.513276    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-nkm7w"]
Oct 30 13:09:15.517063 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.517041    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.543007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.542977    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-nkm7w"]
Oct 30 13:09:15.592632 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.592543    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-webhook-cert\") pod \"maas-controller-5444d7fdb7-nkm7w\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") " pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.592632 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.592586    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-metrics-certs\") pod \"maas-controller-5444d7fdb7-nkm7w\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") " pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.592819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.592683    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdmbd\" (UniqueName: \"kubernetes.io/projected/81b244f2-6fc5-45b9-ba63-08427c71baed-kube-api-access-hdmbd\") pod \"maas-controller-5444d7fdb7-nkm7w\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") " pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.693944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.693907    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-webhook-cert\") pod \"maas-controller-5444d7fdb7-nkm7w\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") " pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.693944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.693944    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-metrics-certs\") pod \"maas-controller-5444d7fdb7-nkm7w\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") " pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.694165 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.694006    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdmbd\" (UniqueName: \"kubernetes.io/projected/81b244f2-6fc5-45b9-ba63-08427c71baed-kube-api-access-hdmbd\") pod \"maas-controller-5444d7fdb7-nkm7w\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") " pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.696634 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.696606    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-webhook-cert\") pod \"maas-controller-5444d7fdb7-nkm7w\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") " pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.696775 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.696606    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-metrics-certs\") pod \"maas-controller-5444d7fdb7-nkm7w\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") " pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.716867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.716836    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdmbd\" (UniqueName: \"kubernetes.io/projected/81b244f2-6fc5-45b9-ba63-08427c71baed-kube-api-access-hdmbd\") pod \"maas-controller-5444d7fdb7-nkm7w\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") " pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:15.795046 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.795003    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.797496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.797469    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert\") pod \"maas-controller-577f596567-6xv5m\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") " pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:15.830444 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:15.830410    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:16.013400 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:16.013366    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:16.129528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:16.129429    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-nkm7w"]
Oct 30 13:09:16.134245 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:09:16.134219    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b244f2_6fc5_45b9_ba63_08427c71baed.slice/crio-3e368bc78a9492d84f62e2ddbafa12179923b997c08aefb5a639b843c02d4ec5 WatchSource:0}: Error finding container 3e368bc78a9492d84f62e2ddbafa12179923b997c08aefb5a639b843c02d4ec5: Status 404 returned error can't find the container with id 3e368bc78a9492d84f62e2ddbafa12179923b997c08aefb5a639b843c02d4ec5
Oct 30 13:09:16.311707 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:16.311677    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-6xv5m"]
Oct 30 13:09:16.316750 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:09:16.316724    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd033bacb_931e_4d59_bc4e_1b1685b379e2.slice/crio-54bd69b15b190c409a51804d7f095555bfbae6137f71e226c8edc9cbec8f8ee7 WatchSource:0}: Error finding container 54bd69b15b190c409a51804d7f095555bfbae6137f71e226c8edc9cbec8f8ee7: Status 404 returned error can't find the container with id 54bd69b15b190c409a51804d7f095555bfbae6137f71e226c8edc9cbec8f8ee7
Oct 30 13:09:16.773446 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:16.773404    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-6xv5m" event={"ID":"d033bacb-931e-4d59-bc4e-1b1685b379e2","Type":"ContainerStarted","Data":"54bd69b15b190c409a51804d7f095555bfbae6137f71e226c8edc9cbec8f8ee7"}
Oct 30 13:09:16.775005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:16.774971    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w" event={"ID":"81b244f2-6fc5-45b9-ba63-08427c71baed","Type":"ContainerStarted","Data":"3e368bc78a9492d84f62e2ddbafa12179923b997c08aefb5a639b843c02d4ec5"}
Oct 30 13:09:19.234215 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:19.234132    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:19.400228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:19.400194    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:19.793257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:19.793223    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-6xv5m" event={"ID":"d033bacb-931e-4d59-bc4e-1b1685b379e2","Type":"ContainerStarted","Data":"8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2"}
Oct 30 13:09:19.793435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:19.793337    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:19.794875 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:19.794848    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w" event={"ID":"81b244f2-6fc5-45b9-ba63-08427c71baed","Type":"ContainerStarted","Data":"4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c"}
Oct 30 13:09:19.794998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:19.794987    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:19.836239 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:19.836182    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-577f596567-6xv5m" podStartSLOduration=2.054156883 podStartE2EDuration="4.836167516s" totalImagesPullingTime="2.782010633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:19.835489207 +0000 UTC m=+697.132706750" watchObservedRunningTime="2026-10-30 13:09:19.836167516 +0000 UTC m=+697.133385056"
Oct 30 13:09:19.887482 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:19.887429    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w" podStartSLOduration=2.090594976 podStartE2EDuration="4.887415683s" totalImagesPullingTime="2.796820707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:19.886872339 +0000 UTC m=+697.184089882" watchObservedRunningTime="2026-10-30 13:09:19.887415683 +0000 UTC m=+697.184633225"
Oct 30 13:09:22.681503 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.681466    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"]
Oct 30 13:09:22.685219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.685196    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.704385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.704359    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 13:09:22.715019 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.714996    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 13:09:22.715019 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.714992    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-hns86\""
Oct 30 13:09:22.715169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.715004    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 13:09:22.718089 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.718065    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"]
Oct 30 13:09:22.762270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.762233    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/dba49010-425f-4862-80ab-887a9930b165-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.762270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.762271    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dba49010-425f-4862-80ab-887a9930b165-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.762543 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.762439    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk9p8\" (UniqueName: \"kubernetes.io/projected/dba49010-425f-4862-80ab-887a9930b165-kube-api-access-vk9p8\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.762543 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.762481    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dba49010-425f-4862-80ab-887a9930b165-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.831727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.831687    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:22.863635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.863599    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/dba49010-425f-4862-80ab-887a9930b165-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.863812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.863640    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dba49010-425f-4862-80ab-887a9930b165-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.863812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.863689    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk9p8\" (UniqueName: \"kubernetes.io/projected/dba49010-425f-4862-80ab-887a9930b165-kube-api-access-vk9p8\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.863812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.863710    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dba49010-425f-4862-80ab-887a9930b165-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.864484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.864455    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/dba49010-425f-4862-80ab-887a9930b165-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.864816 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.864800    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/dba49010-425f-4862-80ab-887a9930b165-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.866184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.866167    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dba49010-425f-4862-80ab-887a9930b165-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.887566 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.887535    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk9p8\" (UniqueName: \"kubernetes.io/projected/dba49010-425f-4862-80ab-887a9930b165-kube-api-access-vk9p8\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-pns2g\" (UID: \"dba49010-425f-4862-80ab-887a9930b165\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.997802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.997708    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:22.998196 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:22.998169    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:23.303161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:23.303077    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"]
Oct 30 13:09:23.308678 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:09:23.308649    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba49010_425f_4862_80ab_887a9930b165.slice/crio-0081dceee86ed2b60ee2f0de6f0d83aa52ff8624036c84061f64946f2f5272b9 WatchSource:0}: Error finding container 0081dceee86ed2b60ee2f0de6f0d83aa52ff8624036c84061f64946f2f5272b9: Status 404 returned error can't find the container with id 0081dceee86ed2b60ee2f0de6f0d83aa52ff8624036c84061f64946f2f5272b9
Oct 30 13:09:23.820374 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:23.820335    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g" event={"ID":"dba49010-425f-4862-80ab-887a9930b165","Type":"ContainerStarted","Data":"0081dceee86ed2b60ee2f0de6f0d83aa52ff8624036c84061f64946f2f5272b9"}
Oct 30 13:09:25.178600 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:25.178506    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:30.801037 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:30.801002    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:30.801657 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:30.801538    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:09:30.922433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:30.922397    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-6xv5m"]
Oct 30 13:09:30.922696 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:30.922649    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-577f596567-6xv5m" podUID="d033bacb-931e-4d59-bc4e-1b1685b379e2" containerName="manager" containerID="cri-o://8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2" gracePeriod=30
Oct 30 13:09:31.182747 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.182718    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:31.253963 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.253932    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert\") pod \"d033bacb-931e-4d59-bc4e-1b1685b379e2\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") "
Oct 30 13:09:31.254157 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.253972    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-metrics-certs\") pod \"d033bacb-931e-4d59-bc4e-1b1685b379e2\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") "
Oct 30 13:09:31.254157 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.254008    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d033bacb-931e-4d59-bc4e-1b1685b379e2-kube-api-access-mjlw7\" (UniqueName: \"kubernetes.io/projected/d033bacb-931e-4d59-bc4e-1b1685b379e2-kube-api-access-mjlw7\") pod \"d033bacb-931e-4d59-bc4e-1b1685b379e2\" (UID: \"d033bacb-931e-4d59-bc4e-1b1685b379e2\") "
Oct 30 13:09:31.256522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.256492    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert" pod "d033bacb-931e-4d59-bc4e-1b1685b379e2" (UID: "d033bacb-931e-4d59-bc4e-1b1685b379e2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:09:31.256644 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.256568    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-metrics-certs" pod "d033bacb-931e-4d59-bc4e-1b1685b379e2" (UID: "d033bacb-931e-4d59-bc4e-1b1685b379e2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:09:31.257693 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.257672    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d033bacb-931e-4d59-bc4e-1b1685b379e2-kube-api-access-mjlw7" pod "d033bacb-931e-4d59-bc4e-1b1685b379e2" (UID: "d033bacb-931e-4d59-bc4e-1b1685b379e2"). InnerVolumeSpecName "kube-api-access-mjlw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:31.355082 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.354992    5781 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-webhook-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:09:31.355082 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.355028    5781 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d033bacb-931e-4d59-bc4e-1b1685b379e2-metrics-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:09:31.355082 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.355039    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjlw7\" (UniqueName: \"kubernetes.io/projected/d033bacb-931e-4d59-bc4e-1b1685b379e2-kube-api-access-mjlw7\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:09:31.871575 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.871538    5781 generic.go:356] "Generic (PLEG): container finished" podID="d033bacb-931e-4d59-bc4e-1b1685b379e2" containerID="8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2" exitCode=0
Oct 30 13:09:31.872049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.871588    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-6xv5m"
Oct 30 13:09:31.872049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.871613    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-6xv5m" event={"ID":"d033bacb-931e-4d59-bc4e-1b1685b379e2","Type":"ContainerDied","Data":"8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2"}
Oct 30 13:09:31.872049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.871668    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-6xv5m" event={"ID":"d033bacb-931e-4d59-bc4e-1b1685b379e2","Type":"ContainerDied","Data":"54bd69b15b190c409a51804d7f095555bfbae6137f71e226c8edc9cbec8f8ee7"}
Oct 30 13:09:31.872049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.871696    5781 scope.go:122] "RemoveContainer" containerID="8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2"
Oct 30 13:09:31.885356 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.885313    5781 scope.go:122] "RemoveContainer" containerID="8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2"
Oct 30 13:09:31.885642 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:09:31.885620    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2\": container with ID starting with 8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2 not found: ID does not exist" containerID="8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2"
Oct 30 13:09:31.885735 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.885648    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2"} err="failed to get container status \"8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2\": rpc error: code = NotFound desc = could not find container \"8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2\": container with ID starting with 8d40413f5a4618b4675a2a08522edbdd3689e20dd44b6689a840013813834dc2 not found: ID does not exist"
Oct 30 13:09:31.924277 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.924212    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-6xv5m"]
Oct 30 13:09:31.935378 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:31.935344    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-577f596567-6xv5m"]
Oct 30 13:09:32.001145 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:09:32.001105    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:09:32.841589 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:32.841560    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d033bacb-931e-4d59-bc4e-1b1685b379e2" path="/var/lib/kubelet/pods/d033bacb-931e-4d59-bc4e-1b1685b379e2/volumes"
Oct 30 13:09:33.883104 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:33.883066    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g" event={"ID":"dba49010-425f-4862-80ab-887a9930b165","Type":"ContainerStarted","Data":"725050f0a007e00a034d8795b403765ebffec565a7e556deffcd62389e78fae0"}
Oct 30 13:09:33.883529 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:33.883177    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:33.926514 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:33.926463    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g" podStartSLOduration=1.530056454 podStartE2EDuration="11.92644639s" totalImagesPullingTime="10.396389936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:33.925678502 +0000 UTC m=+711.222896045" watchObservedRunningTime="2026-10-30 13:09:33.92644639 +0000 UTC m=+711.223663930"
Oct 30 13:09:42.694977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:42.694940    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:44.591263 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:44.591225    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:44.893642 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:44.893569    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-pns2g"
Oct 30 13:09:50.839156 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:50.839123    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:51.027946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:51.027913    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:52.260099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:52.260025    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:09:52.406299 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:09:52.406269    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:10:00.455085 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:00.455049    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:10:01.996038 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:10:01.996000    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:10:30.204505 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:30.204462    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:10:31.996203 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:10:31.996167    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:10:36.876189 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:36.876153    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:10:37.030553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:37.030519    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:10:55.261880 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.261843    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"]
Oct 30 13:10:55.262520 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.262495    5781 cpu_manager.go:414] "containerMap: removing container" podUID="d033bacb-931e-4d59-bc4e-1b1685b379e2" containerName="manager"
Oct 30 13:10:55.262725 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.262698    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d033bacb-931e-4d59-bc4e-1b1685b379e2" containerName="manager"
Oct 30 13:10:55.262908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.262893    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="d033bacb-931e-4d59-bc4e-1b1685b379e2" containerName="manager"
Oct 30 13:10:55.266718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.266697    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.293690 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.293662    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nwjwf\""
Oct 30 13:10:55.293829 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.293662    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 13:10:55.293829 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.293721    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 13:10:55.293829 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.293770    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 30 13:10:55.295434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.295413    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"]
Oct 30 13:10:55.328020 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.327989    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.328169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.328035    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80c5e747-d78e-4007-bbae-5f3e9f47f303-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.328169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.328123    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lctkx\" (UniqueName: \"kubernetes.io/projected/80c5e747-d78e-4007-bbae-5f3e9f47f303-kube-api-access-lctkx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.328169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.328150    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.328317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.328218    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.328317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.328245    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.387488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.387451    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:10:55.428858 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.428817    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.429064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.428863    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.429064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.428917    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.429064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.428964    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80c5e747-d78e-4007-bbae-5f3e9f47f303-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.429064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.428993    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lctkx\" (UniqueName: \"kubernetes.io/projected/80c5e747-d78e-4007-bbae-5f3e9f47f303-kube-api-access-lctkx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.429064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.429034    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.429369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.429345    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.429434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.429373    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.429488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.429427    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.431610 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.431576    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80c5e747-d78e-4007-bbae-5f3e9f47f303-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.431737 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.431614    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80c5e747-d78e-4007-bbae-5f3e9f47f303-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.453643 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.453612    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lctkx\" (UniqueName: \"kubernetes.io/projected/80c5e747-d78e-4007-bbae-5f3e9f47f303-kube-api-access-lctkx\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl\" (UID: \"80c5e747-d78e-4007-bbae-5f3e9f47f303\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.578530 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.578443    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:55.886327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:55.886244    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"]
Oct 30 13:10:55.891934 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:10:55.891905    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80c5e747_d78e_4007_bbae_5f3e9f47f303.slice/crio-8a128504f8f2843188083473169b4f4ebd0e0f013597fffd9dc4f7ecc55ed6c1 WatchSource:0}: Error finding container 8a128504f8f2843188083473169b4f4ebd0e0f013597fffd9dc4f7ecc55ed6c1: Status 404 returned error can't find the container with id 8a128504f8f2843188083473169b4f4ebd0e0f013597fffd9dc4f7ecc55ed6c1
Oct 30 13:10:56.250560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:56.250515    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" event={"ID":"80c5e747-d78e-4007-bbae-5f3e9f47f303","Type":"ContainerStarted","Data":"8a128504f8f2843188083473169b4f4ebd0e0f013597fffd9dc4f7ecc55ed6c1"}
Oct 30 13:10:56.981329 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:56.981303    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"]
Oct 30 13:10:56.986340 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:56.986323    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.003497 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.003454    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 13:10:57.016183 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.016156    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"]
Oct 30 13:10:57.046482 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.046437    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.046648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.046494    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.046648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.046557    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6rxn\" (UniqueName: \"kubernetes.io/projected/ba75e294-c7ff-4681-b4a2-f1892bded953-kube-api-access-s6rxn\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.046733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.046692    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba75e294-c7ff-4681-b4a2-f1892bded953-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.046733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.046722    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.046803 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.046768    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.102699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.102649    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:10:57.147729 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.147699    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba75e294-c7ff-4681-b4a2-f1892bded953-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.147919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.147741    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.147919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.147780    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.147919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.147817    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.147919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.147838    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.147919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.147865    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6rxn\" (UniqueName: \"kubernetes.io/projected/ba75e294-c7ff-4681-b4a2-f1892bded953-kube-api-access-s6rxn\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.148253 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.148216    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.148410 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.148381    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.148510 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.148421    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.150041 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.150024    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba75e294-c7ff-4681-b4a2-f1892bded953-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.150252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.150234    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba75e294-c7ff-4681-b4a2-f1892bded953-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.180889 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.180857    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6rxn\" (UniqueName: \"kubernetes.io/projected/ba75e294-c7ff-4681-b4a2-f1892bded953-kube-api-access-s6rxn\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7\" (UID: \"ba75e294-c7ff-4681-b4a2-f1892bded953\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.257059 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.256956    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" event={"ID":"80c5e747-d78e-4007-bbae-5f3e9f47f303","Type":"ContainerStarted","Data":"5ef36df4a733043c6b53209e310d4f3bc7ef5d92c3c3ab795beb4f1519bb2f96"}
Oct 30 13:10:57.311181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.311141    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:57.315412 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.315369    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podStartSLOduration=1.244536315 podStartE2EDuration="2.315354308s" totalImagesPullingTime="1.070817993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:10:57.314576835 +0000 UTC m=+794.611794378" watchObservedRunningTime="2026-10-30 13:10:57.315354308 +0000 UTC m=+794.612571850"
Oct 30 13:10:57.579202 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.579126    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:10:57.611064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:57.611031    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"]
Oct 30 13:10:57.616050 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:10:57.616020    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba75e294_c7ff_4681_b4a2_f1892bded953.slice/crio-fbcd147d149dc32ba691df05cf0482d88b0568e7f41433ac3691b8b7b31babf1 WatchSource:0}: Error finding container fbcd147d149dc32ba691df05cf0482d88b0568e7f41433ac3691b8b7b31babf1: Status 404 returned error can't find the container with id fbcd147d149dc32ba691df05cf0482d88b0568e7f41433ac3691b8b7b31babf1
Oct 30 13:10:58.262914 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:58.262875    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7" event={"ID":"ba75e294-c7ff-4681-b4a2-f1892bded953","Type":"ContainerStarted","Data":"ae22d4e422f916370b8952fd01ee27cf661f61e9953f3a0df53c1cfe1cc7f890"}
Oct 30 13:10:58.263333 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:58.262930    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7" event={"ID":"ba75e294-c7ff-4681-b4a2-f1892bded953","Type":"ContainerStarted","Data":"fbcd147d149dc32ba691df05cf0482d88b0568e7f41433ac3691b8b7b31babf1"}
Oct 30 13:10:58.305779 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:58.305728    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7" podStartSLOduration=2.068211806 podStartE2EDuration="2.305714925s" totalImagesPullingTime="237.503119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:10:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:10:58.305100396 +0000 UTC m=+795.602317939" watchObservedRunningTime="2026-10-30 13:10:58.305714925 +0000 UTC m=+795.602932461"
Oct 30 13:10:58.311516 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:58.311484    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:10:59.555678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.555612    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"]
Oct 30 13:10:59.559595 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.559564    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.574252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.574231    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 13:10:59.586878 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.586847    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"]
Oct 30 13:10:59.670685 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.670650    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9e82ea8-32e0-4366-bded-499dba1661ec-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.670857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.670698    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.670857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.670726    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.670857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.670800    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.670990 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.670891    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.670990 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.670915    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcpdl\" (UniqueName: \"kubernetes.io/projected/b9e82ea8-32e0-4366-bded-499dba1661ec-kube-api-access-jcpdl\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.681891 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.681862    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:10:59.771431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.771402    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9e82ea8-32e0-4366-bded-499dba1661ec-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.771596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.771442    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.771596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.771464    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.771596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.771481    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.771596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.771536    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.771596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.771558    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcpdl\" (UniqueName: \"kubernetes.io/projected/b9e82ea8-32e0-4366-bded-499dba1661ec-kube-api-access-jcpdl\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.771880 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.771854    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.771941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.771920    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.772034 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.772018    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.773818 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.773798    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9e82ea8-32e0-4366-bded-499dba1661ec-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.774095 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.774075    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9e82ea8-32e0-4366-bded-499dba1661ec-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.799223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.799195    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcpdl\" (UniqueName: \"kubernetes.io/projected/b9e82ea8-32e0-4366-bded-499dba1661ec-kube-api-access-jcpdl\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-86tw4\" (UID: \"b9e82ea8-32e0-4366-bded-499dba1661ec\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:10:59.875491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:10:59.875412    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:11:00.174058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:00.174029    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"]
Oct 30 13:11:00.178585 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:11:00.178556    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e82ea8_32e0_4366_bded_499dba1661ec.slice/crio-cf96237cdfd5d44538eaadb30a9fdf987f55d4391497224c2797a671f2b29579 WatchSource:0}: Error finding container cf96237cdfd5d44538eaadb30a9fdf987f55d4391497224c2797a671f2b29579: Status 404 returned error can't find the container with id cf96237cdfd5d44538eaadb30a9fdf987f55d4391497224c2797a671f2b29579
Oct 30 13:11:00.272672 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:00.272631    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4" event={"ID":"b9e82ea8-32e0-4366-bded-499dba1661ec","Type":"ContainerStarted","Data":"cf96237cdfd5d44538eaadb30a9fdf987f55d4391497224c2797a671f2b29579"}
Oct 30 13:11:00.566512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:00.566432    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:01.278644 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:01.278606    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4" event={"ID":"b9e82ea8-32e0-4366-bded-499dba1661ec","Type":"ContainerStarted","Data":"81daefdc72b377c373db8d3bd2b3ba47a6bc573ccfbdbbab05ac4c55e90ea688"}
Oct 30 13:11:01.324852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:01.324795    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4" podStartSLOduration=1.635960909 podStartE2EDuration="2.324781243s" totalImagesPullingTime="688.820334ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:01.324142788 +0000 UTC m=+798.621360331" watchObservedRunningTime="2026-10-30 13:11:01.324781243 +0000 UTC m=+798.621998785"
Oct 30 13:11:01.875767 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:01.875733    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:11:01.995808 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:11:01.995764    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:11:03.123108 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.123066    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"]
Oct 30 13:11:03.127522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.127497    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.142084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.142061    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 13:11:03.154967 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.154930    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"]
Oct 30 13:11:03.207463 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.207427    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.207648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.207488    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6rdf\" (UniqueName: \"kubernetes.io/projected/906db875-0b0a-41be-9286-7e7de5898576-kube-api-access-v6rdf\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.207648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.207575    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.207726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.207646    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.207726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.207677    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.207726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.207699    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/906db875-0b0a-41be-9286-7e7de5898576-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.229801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.229760    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:03.308974 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.308930    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.309182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.309006    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.309182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.309035    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.309182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.309064    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/906db875-0b0a-41be-9286-7e7de5898576-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.309182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.309110    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.309182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.309167    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6rdf\" (UniqueName: \"kubernetes.io/projected/906db875-0b0a-41be-9286-7e7de5898576-kube-api-access-v6rdf\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.309474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.309429    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.309474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.309447    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.309576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.309503    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.311448 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.311416    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/906db875-0b0a-41be-9286-7e7de5898576-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.311610 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.311588    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/906db875-0b0a-41be-9286-7e7de5898576-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.332050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.332021    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6rdf\" (UniqueName: \"kubernetes.io/projected/906db875-0b0a-41be-9286-7e7de5898576-kube-api-access-v6rdf\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zv422\" (UID: \"906db875-0b0a-41be-9286-7e7de5898576\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.442170 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.442131    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:03.745348 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:03.745231    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"]
Oct 30 13:11:03.750845 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:11:03.750814    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906db875_0b0a_41be_9286_7e7de5898576.slice/crio-ee7fd09028bab465b2591e0a25770c3bd07407473881aef83f077b180a86b0a3 WatchSource:0}: Error finding container ee7fd09028bab465b2591e0a25770c3bd07407473881aef83f077b180a86b0a3: Status 404 returned error can't find the container with id ee7fd09028bab465b2591e0a25770c3bd07407473881aef83f077b180a86b0a3
Oct 30 13:11:04.293461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:04.293366    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422" event={"ID":"906db875-0b0a-41be-9286-7e7de5898576","Type":"ContainerStarted","Data":"d005e64c89a32a87fd0fa2b11b6ba932f00cda9e64a2158a37136373c0842564"}
Oct 30 13:11:04.293461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:04.293420    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422" event={"ID":"906db875-0b0a-41be-9286-7e7de5898576","Type":"ContainerStarted","Data":"ee7fd09028bab465b2591e0a25770c3bd07407473881aef83f077b180a86b0a3"}
Oct 30 13:11:04.338487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:04.338421    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422" podStartSLOduration=1.109313422 podStartE2EDuration="1.338406667s" totalImagesPullingTime="229.093245ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:04.337840443 +0000 UTC m=+801.635057986" watchObservedRunningTime="2026-10-30 13:11:04.338406667 +0000 UTC m=+801.635624209"
Oct 30 13:11:04.443077 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:04.443039    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:05.070936 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:05.070890    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:05.579541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:05.579506    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:11:05.591942 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:05.591914    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:11:06.313849 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:06.313820    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:11:07.312090 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:07.312060    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:11:07.324986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:07.324960    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:11:08.041941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:08.041908    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:08.323809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:08.323731    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7"
Oct 30 13:11:09.876431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:09.876396    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:11:09.889145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:09.889120    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:11:10.333492 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:10.333464    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-86tw4"
Oct 30 13:11:12.544404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.544363    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"]
Oct 30 13:11:12.548825 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.548785    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.571524 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.571493    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Oct 30 13:11:12.582301 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.582259    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"]
Oct 30 13:11:12.598173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.598145    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.598368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.598177    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98ppv\" (UniqueName: \"kubernetes.io/projected/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-kube-api-access-98ppv\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.598368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.598300    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.598486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.598392    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.598486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.598411    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.598596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.598512    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.649121 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.649088    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:12.699498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.699456    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.699498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.699497    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98ppv\" (UniqueName: \"kubernetes.io/projected/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-kube-api-access-98ppv\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.699759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.699556    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.699759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.699596    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.699759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.699612    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.699759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.699680    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.699963 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.699927    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.700181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.700153    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.700181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.700179    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.702050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.702025    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.702279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.702261    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.732609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.732581    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98ppv\" (UniqueName: \"kubernetes.io/projected/74c20097-c1fa-4692-b0f6-5b69f8c6ee31-kube-api-access-98ppv\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n\" (UID: \"74c20097-c1fa-4692-b0f6-5b69f8c6ee31\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:12.862495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:12.862408    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:13.172765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:13.172688    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"]
Oct 30 13:11:13.177481 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:11:13.177456    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74c20097_c1fa_4692_b0f6_5b69f8c6ee31.slice/crio-8cdae5d9cdef6a02a1e80e85934c5a7a7febf6ff25aba4b793092729cd37af2c WatchSource:0}: Error finding container 8cdae5d9cdef6a02a1e80e85934c5a7a7febf6ff25aba4b793092729cd37af2c: Status 404 returned error can't find the container with id 8cdae5d9cdef6a02a1e80e85934c5a7a7febf6ff25aba4b793092729cd37af2c
Oct 30 13:11:13.337912 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:13.337875    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n" event={"ID":"74c20097-c1fa-4692-b0f6-5b69f8c6ee31","Type":"ContainerStarted","Data":"8cdae5d9cdef6a02a1e80e85934c5a7a7febf6ff25aba4b793092729cd37af2c"}
Oct 30 13:11:13.443154 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:13.443116    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:13.456008 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:13.455985    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:14.344359 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:14.344319    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n" event={"ID":"74c20097-c1fa-4692-b0f6-5b69f8c6ee31","Type":"ContainerStarted","Data":"f5abf53446f8aa23254930a7dfe7413020cb5f9d9c822a237a6cd76819f6bf9f"}
Oct 30 13:11:14.355677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:14.355653    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zv422"
Oct 30 13:11:14.390573 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:14.390525    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n" podStartSLOduration=2.180363453 podStartE2EDuration="2.390510061s" totalImagesPullingTime="210.146608ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:14.389837008 +0000 UTC m=+811.687054577" watchObservedRunningTime="2026-10-30 13:11:14.390510061 +0000 UTC m=+811.687727601"
Oct 30 13:11:14.863003 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:14.862965    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:18.742359 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.742320    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"]
Oct 30 13:11:18.746409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.746389    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.762399 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.762363    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 13:11:18.776700 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.776672    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"]
Oct 30 13:11:18.845229 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.845205    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5td7d\""
Oct 30 13:11:18.862279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.862235    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.862279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.862297    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.862511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.862366    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/035c8706-673c-4624-8b95-57c566fba8db-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.862511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.862432    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.862511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.862463    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cgxr\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-kube-api-access-6cgxr\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.862511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.862492    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.862634 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.862595    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/035c8706-673c-4624-8b95-57c566fba8db-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.862665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.862633    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.862713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.862697    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.963724 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.963685    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/035c8706-673c-4624-8b95-57c566fba8db-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.963724 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.963733    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.963973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.963781    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.963973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.963800    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.963973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.963820    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.963973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.963837    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/035c8706-673c-4624-8b95-57c566fba8db-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.963973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.963866    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.963973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.963902    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cgxr\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-kube-api-access-6cgxr\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.963973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.963965    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.964357 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.964262    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.964357 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.964281    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.964478 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.964370    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.964478 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.964438    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.965201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.965171    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/035c8706-673c-4624-8b95-57c566fba8db-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.966254 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.966225    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.966606 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.966584    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/035c8706-673c-4624-8b95-57c566fba8db-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.987369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.987335    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cgxr\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-kube-api-access-6cgxr\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:18.992007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:18.991980    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-5r99l\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:19.090253 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:19.090170    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:19.406732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:19.406642    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"]
Oct 30 13:11:19.410876 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:11:19.410848    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035c8706_673c_4624_8b95_57c566fba8db.slice/crio-8fbea125afc3bc4e6a9a28e90d13950cf4fe59318b092ef13d7b4ef1a7dd278a WatchSource:0}: Error finding container 8fbea125afc3bc4e6a9a28e90d13950cf4fe59318b092ef13d7b4ef1a7dd278a: Status 404 returned error can't find the container with id 8fbea125afc3bc4e6a9a28e90d13950cf4fe59318b092ef13d7b4ef1a7dd278a
Oct 30 13:11:20.380880 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:20.380841    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" event={"ID":"035c8706-673c-4624-8b95-57c566fba8db","Type":"ContainerStarted","Data":"8fbea125afc3bc4e6a9a28e90d13950cf4fe59318b092ef13d7b4ef1a7dd278a"}
Oct 30 13:11:21.902581 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:21.902547    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:11:21.902898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:21.902612    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:11:21.902898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:21.902640    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:11:22.391985 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:22.391944    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" event={"ID":"035c8706-673c-4624-8b95-57c566fba8db","Type":"ContainerStarted","Data":"1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0"}
Oct 30 13:11:22.449663 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:22.449609    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" podStartSLOduration=1.960830873 podStartE2EDuration="4.449595976s" totalImagesPullingTime="2.488765103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:22.448280911 +0000 UTC m=+819.745498453" watchObservedRunningTime="2026-10-30 13:11:22.449595976 +0000 UTC m=+819.746813519"
Oct 30 13:11:22.863480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:22.863441    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:22.876062 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:22.876031    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:23.091372 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:23.091338    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:23.093009 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:23.092979    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-5r99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 13:11:23.093151 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:23.093053    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 13:11:23.407165 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:23.407135    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n"
Oct 30 13:11:24.091351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:24.091316    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-5r99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 13:11:24.091537 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:24.091368    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 13:11:25.091089 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:25.091052    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-5r99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 13:11:25.091330 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:25.091111    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 13:11:26.091270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:26.091231    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-5r99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 13:11:26.091689 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:26.091323    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 13:11:27.090638 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:27.090602    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-5r99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 13:11:27.090821 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:27.090661    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 13:11:28.091430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:28.091388    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-5r99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 13:11:28.091832 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:28.091449    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 13:11:29.091042 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:29.091001    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-5r99l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Oct 30 13:11:29.091219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:29.091065    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Oct 30 13:11:29.871667 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:29.871623    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:30.018143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:30.018106    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:30.156409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:30.156323    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:30.156766 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:30.156741    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:30.157516 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:30.157495    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:11:30.340539 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:30.340496    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:31.996926 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:11:31.996770    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:11:33.773007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.772970    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"]
Oct 30 13:11:33.777009 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.776982    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.813182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.813145    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"]
Oct 30 13:11:33.840791 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.840758    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rcrt\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-kube-api-access-6rcrt\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.840954 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.840798    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.840954 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.840904    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.840954 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.840931    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65918173-d800-4142-9f9a-fe8d32d5504d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.841060 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.840964    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.841060 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.841037    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.841125 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.841077    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.841125 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.841116    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65918173-d800-4142-9f9a-fe8d32d5504d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.841186 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.841153    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.941758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.941719    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.941758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.941756    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65918173-d800-4142-9f9a-fe8d32d5504d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.941809    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.941839    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.941869    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.941898    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65918173-d800-4142-9f9a-fe8d32d5504d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.941948    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.942063    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rcrt\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-kube-api-access-6rcrt\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.942096    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.942306    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.942384    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942597 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.942552    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942597 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.942564    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65918173-d800-4142-9f9a-fe8d32d5504d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.942688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.942616    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.944394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.944365    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.944600 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.944578    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65918173-d800-4142-9f9a-fe8d32d5504d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.967330 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.967300    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:33.967507 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:33.967487    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rcrt\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-kube-api-access-6rcrt\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zt4gh\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:34.090342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:34.090228    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:34.410669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:34.410573    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"]
Oct 30 13:11:34.415249 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:11:34.415217    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65918173_d800_4142_9f9a_fe8d32d5504d.slice/crio-3d4eaa53a5b0551e707e07c84e2510cd7aec2b82df5127618689eab181d4bd68 WatchSource:0}: Error finding container 3d4eaa53a5b0551e707e07c84e2510cd7aec2b82df5127618689eab181d4bd68: Status 404 returned error can't find the container with id 3d4eaa53a5b0551e707e07c84e2510cd7aec2b82df5127618689eab181d4bd68
Oct 30 13:11:34.418622 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:34.418588    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:11:34.418736 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:34.418658    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:11:34.418736 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:34.418686    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:11:34.486532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:34.486486    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" event={"ID":"65918173-d800-4142-9f9a-fe8d32d5504d","Type":"ContainerStarted","Data":"db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea"}
Oct 30 13:11:34.486719 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:34.486575    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" event={"ID":"65918173-d800-4142-9f9a-fe8d32d5504d","Type":"ContainerStarted","Data":"3d4eaa53a5b0551e707e07c84e2510cd7aec2b82df5127618689eab181d4bd68"}
Oct 30 13:11:34.538444 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:34.538342    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podStartSLOduration=1.5383255390000001 podStartE2EDuration="1.538325539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:34.53653008 +0000 UTC m=+831.833747622" watchObservedRunningTime="2026-10-30 13:11:34.538325539 +0000 UTC m=+831.835543082"
Oct 30 13:11:35.090892 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:35.090856    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:35.092449 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:35.092421    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 13:11:35.092591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:35.092476    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 13:11:35.505644 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:35.505615    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:35.675685 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:35.675639    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:11:36.091597 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:36.091560    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 13:11:36.091985 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:36.091621    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 13:11:37.090955 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:37.090921    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 13:11:37.091148 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:37.090976    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 13:11:38.091524 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:38.091492    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 13:11:38.091914 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:38.091548    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 13:11:39.090978 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:39.090937    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 13:11:39.091199 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:39.091016    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 13:11:40.091141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:40.091104    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 13:11:40.091552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:40.091171    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 13:11:41.090908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:41.090872    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Oct 30 13:11:41.091094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:41.090926    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Oct 30 13:11:43.091515 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:43.091469    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:11:43.092039 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:43.091543    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:11:44.132514 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:44.132466    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:11:44.133034 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:44.132544    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:11:44.826856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:44.826829    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:44.827155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:44.827127    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:44.827702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:44.827686    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:11:48.854490 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.854446    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"]
Oct 30 13:11:48.859387 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.859368    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:48.912158 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.912123    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"]
Oct 30 13:11:48.997764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.997725    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:48.997939 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.997844    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zr9h\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-kube-api-access-8zr9h\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:48.997939 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.997884    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:48.998011 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.997969    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc834143-970e-4e23-8140-fc53aec8ea54-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:48.998011 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.997991    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:48.998076 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.998014    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:48.998107 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.998082    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:48.998142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.998111    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:48.998178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:48.998146    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc834143-970e-4e23-8140-fc53aec8ea54-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.099486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.099446    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.099486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.099494    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.099713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.099513    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.099713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.099531    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc834143-970e-4e23-8140-fc53aec8ea54-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.099713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.099573    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.099713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.099629    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zr9h\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-kube-api-access-8zr9h\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.099713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.099662    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.099713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.099712    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc834143-970e-4e23-8140-fc53aec8ea54-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.100003 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.099743    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.100133 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.100077    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.100319 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.100268    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.100383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.100344    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.100419 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.100383    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc834143-970e-4e23-8140-fc53aec8ea54-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.100452 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.100422    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.101848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.101832    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.102162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.102142    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc834143-970e-4e23-8140-fc53aec8ea54-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.131002 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.130927    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zr9h\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-kube-api-access-8zr9h\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.131162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.131138    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-6797c\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.172518 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.172482    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:49.491739 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.491702    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"]
Oct 30 13:11:49.495129 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:11:49.495100    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc834143_970e_4e23_8140_fc53aec8ea54.slice/crio-36d0234ded1abe36f890bcc212234b7d19085b4d530482f20d58a7f529f236a0 WatchSource:0}: Error finding container 36d0234ded1abe36f890bcc212234b7d19085b4d530482f20d58a7f529f236a0: Status 404 returned error can't find the container with id 36d0234ded1abe36f890bcc212234b7d19085b4d530482f20d58a7f529f236a0
Oct 30 13:11:49.497331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.497301    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:11:49.497421 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.497364    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:11:49.497421 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.497392    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:11:49.561110 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.561080    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" event={"ID":"fc834143-970e-4e23-8140-fc53aec8ea54","Type":"ContainerStarted","Data":"c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d"}
Oct 30 13:11:49.561238 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.561137    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" event={"ID":"fc834143-970e-4e23-8140-fc53aec8ea54","Type":"ContainerStarted","Data":"36d0234ded1abe36f890bcc212234b7d19085b4d530482f20d58a7f529f236a0"}
Oct 30 13:11:49.608243 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:49.608196    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" podStartSLOduration=1.608181159 podStartE2EDuration="1.608181159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:49.605108221 +0000 UTC m=+846.902325790" watchObservedRunningTime="2026-10-30 13:11:49.608181159 +0000 UTC m=+846.905398701"
Oct 30 13:11:50.173033 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:50.173000    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:50.174646 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:50.174623    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6797c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 13:11:50.174765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:50.174665    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 13:11:51.173659 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:51.173618    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6797c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 13:11:51.174075 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:51.173672    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 13:11:52.172950 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:52.172912    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6797c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 13:11:52.173146 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:52.172972    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 13:11:53.172887 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:53.172855    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6797c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 13:11:53.173368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:53.172975    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 13:11:54.173176 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:54.173141    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6797c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 13:11:54.173606 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:54.173223    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 13:11:55.173021 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:55.172984    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6797c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 13:11:55.173212 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:55.173051    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 13:11:56.173011 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:56.172964    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-6797c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Oct 30 13:11:56.173187 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:56.173052    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Oct 30 13:11:57.755009 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:57.754969    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:57.755475 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:57.755433    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:11:57.756352 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:11:57.756332    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:12:00.710641 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:00.710610    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:01.995966 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:12:01.995931    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:12:29.336033 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:29.336003    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:29.971610 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:29.971572    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:30.430425 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:30.430400    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:31.648012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:31.647980    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:31.826209 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:31.826177    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:31.996891 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:12:31.996852    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:12:33.358521 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:33.358490    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:33.548203 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:33.548162    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:37.637543 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:37.637509    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:38.678105 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:38.678064    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:12:42.928367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:42.928336    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:12:42.929619 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:12:42.929599    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:13:00.878028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:00.877944    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:01.996398 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:13:01.996366    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:13:04.738523 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:04.738486    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:04.889616 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:04.889578    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:05.185823 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.185791    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"]
Oct 30 13:13:05.189632 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.189607    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.212185 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.212157    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-031737-openshift-default-dockercfg-77v2d\""
Oct 30 13:13:05.217729 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.217700    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.217862 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.217781    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.217906 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.217871    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.217906 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.217898    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.218076 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.218056    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.218125 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.218112    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519f908a-bf41-4285-8ac4-1abe0a9d893e-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.218171 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.218143    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.218353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.218269    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87phn\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-kube-api-access-87phn\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.218353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.218335    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.230425 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.230399    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"]
Oct 30 13:13:05.319570 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.319535    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.319769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.319583    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519f908a-bf41-4285-8ac4-1abe0a9d893e-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.319769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.319601    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.319769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.319644    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87phn\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-kube-api-access-87phn\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.319769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.319667    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.319769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.319693    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.319769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.319755    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.320088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.319798    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.320088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.319828    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.320088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.320025    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.320088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.320078    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.320319 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.320264    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.320382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.320355    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.320657 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.320636    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519f908a-bf41-4285-8ac4-1abe0a9d893e-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.322049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.322029    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.322156 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.322137    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.351919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.351787    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.361679 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.361649    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87phn\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-kube-api-access-87phn\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.403531 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.403496    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:05.504497 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.504422    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:05.622522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.622488    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:05.787276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.787194    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:05.830500 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.830463    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"]
Oct 30 13:13:05.835183 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:05.835155    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519f908a_bf41_4285_8ac4_1abe0a9d893e.slice/crio-32d725537e5c4d01d8dcc2d7b853a43695f2fe0d6a1969a8eac70e0069731443 WatchSource:0}: Error finding container 32d725537e5c4d01d8dcc2d7b853a43695f2fe0d6a1969a8eac70e0069731443: Status 404 returned error can't find the container with id 32d725537e5c4d01d8dcc2d7b853a43695f2fe0d6a1969a8eac70e0069731443
Oct 30 13:13:05.837760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.837730    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:05.837858 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.837793    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:05.837858 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.837842    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:05.851906 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.851851    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"]
Oct 30 13:13:05.857479 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.857458    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.893512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.893487    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"]
Oct 30 13:13:05.924802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.924761    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.924979 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.924831    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.924979 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.924892    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.924979 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.924961    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.925141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.924999    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.925141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.925063    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.925141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.925095    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nfkf\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-kube-api-access-2nfkf\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.925141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.925126    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.925436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.925158    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:05.925519 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.925497    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-05942d-openshift-default-dockercfg-hrnkn\""
Oct 30 13:13:05.926919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.926207    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" event={"ID":"519f908a-bf41-4285-8ac4-1abe0a9d893e","Type":"ContainerStarted","Data":"70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf"}
Oct 30 13:13:05.926919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.926275    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" event={"ID":"519f908a-bf41-4285-8ac4-1abe0a9d893e","Type":"ContainerStarted","Data":"32d725537e5c4d01d8dcc2d7b853a43695f2fe0d6a1969a8eac70e0069731443"}
Oct 30 13:13:05.957111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.957074    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:05.973625 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:05.973579    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" podStartSLOduration=0.973565904 podStartE2EDuration="973.565904ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:05.972515224 +0000 UTC m=+923.269732778" watchObservedRunningTime="2026-10-30 13:13:05.973565904 +0000 UTC m=+923.270783500"
Oct 30 13:13:06.004062 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.004026    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"]
Oct 30 13:13:06.007921 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.007902    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.026750 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.026724    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.026917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.026772    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.026917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.026799    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.026917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.026825    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.026917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.026878    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e520d651-7215-4f0d-9ecd-c8eace846b1a-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.027083 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027064    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.027135 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027119    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027208    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027256    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nfkf\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-kube-api-access-2nfkf\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027485    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027542    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027570    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027604    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbww7\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-kube-api-access-zbww7\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027648    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027657    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027677    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.027746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027858    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.028496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.027947    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.028496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.028028    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.028496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.028067    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.028496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.028118    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.028496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.028351    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.028496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.028406    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.029520 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.029501    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-d0a18c-openshift-default-dockercfg-jqvd8\""
Oct 30 13:13:06.029730 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.029712    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.030740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.030721    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.043517 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.043459    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"]
Oct 30 13:13:06.053248 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.053217    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.053478 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.053403    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nfkf\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-kube-api-access-2nfkf\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.127974 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.127931    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:06.129356 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129329    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.129444 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129413    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.129444 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129439    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.129551 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129466    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.129551 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129505    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e520d651-7215-4f0d-9ecd-c8eace846b1a-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.129664 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129637    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.129759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129690    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbww7\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-kube-api-access-zbww7\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.129759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129751    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.129936 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129795    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.129936 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.129812    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.130422 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.130365    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.130544 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.130472    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.130607 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.130586    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.130853 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.130830    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e520d651-7215-4f0d-9ecd-c8eace846b1a-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.131850 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.131830    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.132355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.132338    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.157979 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.157943    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.171575 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.171550    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbww7\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-kube-api-access-zbww7\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.188512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.188485    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:06.274812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.274775    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:06.325000 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.324899    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:06.423051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.423016    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:06.504718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.504686    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:06.510070 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.510046    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:06.513889 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.513865    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"]
Oct 30 13:13:06.518730 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:06.518706    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd4e4e0f_5c48_4804_ad64_f9121adb82db.slice/crio-1f7bc0b7499bc1d18b77a75af886fb084b583ab82babe13fbfc629493174db08 WatchSource:0}: Error finding container 1f7bc0b7499bc1d18b77a75af886fb084b583ab82babe13fbfc629493174db08: Status 404 returned error can't find the container with id 1f7bc0b7499bc1d18b77a75af886fb084b583ab82babe13fbfc629493174db08
Oct 30 13:13:06.521138 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.521106    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:06.521234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.521181    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:06.521234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.521223    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:06.643440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.643408    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"]
Oct 30 13:13:06.648898 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:06.648872    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode520d651_7215_4f0d_9ecd_c8eace846b1a.slice/crio-1dcccfb0f342998af83e111f8dc78cbf7d8835fe1a6ddcb0c86948219121e28f WatchSource:0}: Error finding container 1dcccfb0f342998af83e111f8dc78cbf7d8835fe1a6ddcb0c86948219121e28f: Status 404 returned error can't find the container with id 1dcccfb0f342998af83e111f8dc78cbf7d8835fe1a6ddcb0c86948219121e28f
Oct 30 13:13:06.651662 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.651629    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:06.651748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.651723    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:06.651801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.651751    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:06.934007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.933969    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" event={"ID":"cd4e4e0f-5c48-4804-ad64-f9121adb82db","Type":"ContainerStarted","Data":"49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a"}
Oct 30 13:13:06.934511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.934028    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" event={"ID":"cd4e4e0f-5c48-4804-ad64-f9121adb82db","Type":"ContainerStarted","Data":"1f7bc0b7499bc1d18b77a75af886fb084b583ab82babe13fbfc629493174db08"}
Oct 30 13:13:06.935592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.935565    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" event={"ID":"e520d651-7215-4f0d-9ecd-c8eace846b1a","Type":"ContainerStarted","Data":"96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e"}
Oct 30 13:13:06.935715 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.935615    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" event={"ID":"e520d651-7215-4f0d-9ecd-c8eace846b1a","Type":"ContainerStarted","Data":"1dcccfb0f342998af83e111f8dc78cbf7d8835fe1a6ddcb0c86948219121e28f"}
Oct 30 13:13:06.935764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.935744    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:06.936655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.936635    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:06.978888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:06.978839    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podStartSLOduration=1.978824553 podStartE2EDuration="1.978824553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:06.978751868 +0000 UTC m=+924.275969410" watchObservedRunningTime="2026-10-30 13:13:06.978824553 +0000 UTC m=+924.276042096"
Oct 30 13:13:07.084871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:07.084302    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" podStartSLOduration=2.084268114 podStartE2EDuration="2.084268114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:07.08240718 +0000 UTC m=+924.379624725" watchObservedRunningTime="2026-10-30 13:13:07.084268114 +0000 UTC m=+924.381485658"
Oct 30 13:13:07.189687 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:07.189610    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:07.194684 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:07.194653    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:07.326750 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:07.326720    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:07.332638 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:07.332611    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:07.941347 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:07.941316    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:07.941855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:07.941376    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:07.942230 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:07.942213    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:07.942440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:07.942422    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:08.805497 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:08.805464    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:08.981462 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:08.981427    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:09.771913 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:09.771873    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:10.021902 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:10.021875    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:10.237506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:10.237465    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:10.450593 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:10.450554    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:10.776983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:10.776945    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:11.302272 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.302232    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:11.456537 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.456503    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"]
Oct 30 13:13:11.461778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.461757    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:11.483338 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.483317    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-031737-dockercfg-bh9nd\""
Oct 30 13:13:11.483338 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.483327    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-031737\""
Oct 30 13:13:11.488844 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.488820    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrbvw\" (UniqueName: \"kubernetes.io/projected/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-kube-api-access-hrbvw\") pod \"payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r\" (UID: \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:11.488934 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.488861    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r\" (UID: \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:11.496818 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.496764    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"]
Oct 30 13:13:11.547327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.536217    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:11.589956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.589870    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrbvw\" (UniqueName: \"kubernetes.io/projected/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-kube-api-access-hrbvw\") pod \"payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r\" (UID: \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:11.589956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.589928    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r\" (UID: \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:11.590716 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.590690    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r\" (UID: \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:11.611994 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.611967    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrbvw\" (UniqueName: \"kubernetes.io/projected/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-kube-api-access-hrbvw\") pod \"payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r\" (UID: \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:11.775861 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:11.775826    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:12.078005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:12.077974    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"]
Oct 30 13:13:12.082160 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:12.082134    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a0f9b16_2f25_4a1a_b7a9_803fa6939b7f.slice/crio-1ba7829c6fc5e51d3f16013776f262f708df967c62c2888ed7b9c57aef12f981 WatchSource:0}: Error finding container 1ba7829c6fc5e51d3f16013776f262f708df967c62c2888ed7b9c57aef12f981: Status 404 returned error can't find the container with id 1ba7829c6fc5e51d3f16013776f262f708df967c62c2888ed7b9c57aef12f981
Oct 30 13:13:12.969732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:12.969697    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r" event={"ID":"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f","Type":"ContainerStarted","Data":"1ba7829c6fc5e51d3f16013776f262f708df967c62c2888ed7b9c57aef12f981"}
Oct 30 13:13:13.234162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.234094    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:13.362127 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.362091    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"]
Oct 30 13:13:13.371562 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.371540    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:13.401868 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.401452    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-88f6a4-dockercfg-7fhkf\""
Oct 30 13:13:13.401868 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.401514    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-88f6a4\""
Oct 30 13:13:13.403850 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.403814    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"]
Oct 30 13:13:13.410305 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.410256    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzlc9\" (UniqueName: \"kubernetes.io/projected/72ed25d8-f936-44c6-9003-79ff1d8da414-kube-api-access-hzlc9\") pod \"payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6\" (UID: \"72ed25d8-f936-44c6-9003-79ff1d8da414\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:13.410445 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.410423    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72ed25d8-f936-44c6-9003-79ff1d8da414-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6\" (UID: \"72ed25d8-f936-44c6-9003-79ff1d8da414\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:13.440327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.437486    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:13.511590 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.511511    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72ed25d8-f936-44c6-9003-79ff1d8da414-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6\" (UID: \"72ed25d8-f936-44c6-9003-79ff1d8da414\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:13.511807 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.511665    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzlc9\" (UniqueName: \"kubernetes.io/projected/72ed25d8-f936-44c6-9003-79ff1d8da414-kube-api-access-hzlc9\") pod \"payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6\" (UID: \"72ed25d8-f936-44c6-9003-79ff1d8da414\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:13.512256 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.512230    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72ed25d8-f936-44c6-9003-79ff1d8da414-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6\" (UID: \"72ed25d8-f936-44c6-9003-79ff1d8da414\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:13.542774 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.542748    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzlc9\" (UniqueName: \"kubernetes.io/projected/72ed25d8-f936-44c6-9003-79ff1d8da414-kube-api-access-hzlc9\") pod \"payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6\" (UID: \"72ed25d8-f936-44c6-9003-79ff1d8da414\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:13.686870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:13.686833    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:14.016508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.016337    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"]
Oct 30 13:13:14.019567 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:14.019541    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ed25d8_f936_44c6_9003_79ff1d8da414.slice/crio-3de367902f89dcb264a660a1952bbde4a2b82a1d05d782c84b1a71fd2c7ef1b6 WatchSource:0}: Error finding container 3de367902f89dcb264a660a1952bbde4a2b82a1d05d782c84b1a71fd2c7ef1b6: Status 404 returned error can't find the container with id 3de367902f89dcb264a660a1952bbde4a2b82a1d05d782c84b1a71fd2c7ef1b6
Oct 30 13:13:14.200051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.200026    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:14.411928 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.411899    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:14.435198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.435172    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"]
Oct 30 13:13:14.439177 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.439160    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:14.454228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.454208    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-d0a18c-dockercfg-rxr9x\""
Oct 30 13:13:14.467090 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.467070    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"]
Oct 30 13:13:14.475635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.475611    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-d0a18c\""
Oct 30 13:13:14.524275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.524189    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q987\" (UniqueName: \"kubernetes.io/projected/95db4022-c4ff-4a65-9142-d336e04cb839-kube-api-access-4q987\") pod \"payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv\" (UID: \"95db4022-c4ff-4a65-9142-d336e04cb839\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:14.524604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.524578    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95db4022-c4ff-4a65-9142-d336e04cb839-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv\" (UID: \"95db4022-c4ff-4a65-9142-d336e04cb839\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:14.578573 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.578543    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:14.626429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.626364    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q987\" (UniqueName: \"kubernetes.io/projected/95db4022-c4ff-4a65-9142-d336e04cb839-kube-api-access-4q987\") pod \"payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv\" (UID: \"95db4022-c4ff-4a65-9142-d336e04cb839\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:14.626429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.626420    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95db4022-c4ff-4a65-9142-d336e04cb839-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv\" (UID: \"95db4022-c4ff-4a65-9142-d336e04cb839\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:14.627138 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.627112    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95db4022-c4ff-4a65-9142-d336e04cb839-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv\" (UID: \"95db4022-c4ff-4a65-9142-d336e04cb839\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:14.649023 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.648999    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q987\" (UniqueName: \"kubernetes.io/projected/95db4022-c4ff-4a65-9142-d336e04cb839-kube-api-access-4q987\") pod \"payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv\" (UID: \"95db4022-c4ff-4a65-9142-d336e04cb839\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:14.752888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.752864    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:14.780568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.780539    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:14.982870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.982836    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r" event={"ID":"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f","Type":"ContainerStarted","Data":"3d451cb5cca7852ed1daa13df4001962c1cc8a3a575f72f36b1590191fc58864"}
Oct 30 13:13:14.983074 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.983057    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:14.984327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.984306    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6" event={"ID":"72ed25d8-f936-44c6-9003-79ff1d8da414","Type":"ContainerStarted","Data":"f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5"}
Oct 30 13:13:14.984430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.984342    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6" event={"ID":"72ed25d8-f936-44c6-9003-79ff1d8da414","Type":"ContainerStarted","Data":"3de367902f89dcb264a660a1952bbde4a2b82a1d05d782c84b1a71fd2c7ef1b6"}
Oct 30 13:13:14.984468 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:14.984434    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:15.046194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.046144    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r" podStartSLOduration=2.051783992 podStartE2EDuration="4.046131168s" totalImagesPullingTime="1.994347176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:15.044973871 +0000 UTC m=+932.342191414" watchObservedRunningTime="2026-10-30 13:13:15.046131168 +0000 UTC m=+932.343348713"
Oct 30 13:13:15.056670 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.056558    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"]
Oct 30 13:13:15.059503 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:15.059472    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95db4022_c4ff_4a65_9142_d336e04cb839.slice/crio-51cc1640cb9b95d028eca6ecdb13788d6973bb9c4ff6e40e4902782777b363e1 WatchSource:0}: Error finding container 51cc1640cb9b95d028eca6ecdb13788d6973bb9c4ff6e40e4902782777b363e1: Status 404 returned error can't find the container with id 51cc1640cb9b95d028eca6ecdb13788d6973bb9c4ff6e40e4902782777b363e1
Oct 30 13:13:15.114433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.114383    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6" podStartSLOduration=1.800670549 podStartE2EDuration="2.114365499s" totalImagesPullingTime="313.69495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:15.111675005 +0000 UTC m=+932.408892548" watchObservedRunningTime="2026-10-30 13:13:15.114365499 +0000 UTC m=+932.411583044"
Oct 30 13:13:15.233753 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.232924    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"]
Oct 30 13:13:15.237459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.237438    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.259471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.259447    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-f76c09-openshift-default-dockercfg-z6xv9\""
Oct 30 13:13:15.281337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.281279    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"]
Oct 30 13:13:15.333717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.333689    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.333717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.333718    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.333913 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.333756    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.333913 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.333840    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.333913 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.333877    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.334011 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.333917    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qwbr\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-kube-api-access-7qwbr\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.334011 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.333993    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.334075 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.334035    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.334075 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.334062    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb08489-1221-4ab8-9b10-df2003f17b23-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.405457 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.405424    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:15.435662 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.435625    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.435819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.435664    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.435819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.435706    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.435819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.435728    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.435819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.435755    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.435819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.435790    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qwbr\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-kube-api-access-7qwbr\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.436099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.435857    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.436099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.435875    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.436099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.435896    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb08489-1221-4ab8-9b10-df2003f17b23-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.436099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.436084    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.436464 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.436441    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.436666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.436646    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb08489-1221-4ab8-9b10-df2003f17b23-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.436762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.436674    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.436762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.436650    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.438080 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.438053    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.438299 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.438270    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.464967 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.464947    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.465541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.465524    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qwbr\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-kube-api-access-7qwbr\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.552781 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.552707    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:15.625933 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.625900    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:15.635974 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.635953    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"]
Oct 30 13:13:15.641259 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.641240    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:15.671964 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.671930    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"]
Oct 30 13:13:15.690094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.687268    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-ab3176\""
Oct 30 13:13:15.690094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.687492    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-ab3176-dockercfg-8bx8s\""
Oct 30 13:13:15.739051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.739015    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgjg5\" (UniqueName: \"kubernetes.io/projected/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-kube-api-access-jgjg5\") pod \"payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7\" (UID: \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:15.739051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.739051    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7\" (UID: \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:15.840481 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.840405    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgjg5\" (UniqueName: \"kubernetes.io/projected/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-kube-api-access-jgjg5\") pod \"payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7\" (UID: \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:15.840481 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.840443    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7\" (UID: \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:15.841159 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.841140    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7\" (UID: \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:15.868270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.868233    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgjg5\" (UniqueName: \"kubernetes.io/projected/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-kube-api-access-jgjg5\") pod \"payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7\" (UID: \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:15.887824 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.887798    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"]
Oct 30 13:13:15.893089 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:15.893063    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb08489_1221_4ab8_9b10_df2003f17b23.slice/crio-3c2dcfeb56f974febfd4526eadccc941af807d2a8093224695f295ea7a0759ab WatchSource:0}: Error finding container 3c2dcfeb56f974febfd4526eadccc941af807d2a8093224695f295ea7a0759ab: Status 404 returned error can't find the container with id 3c2dcfeb56f974febfd4526eadccc941af807d2a8093224695f295ea7a0759ab
Oct 30 13:13:15.895757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.895722    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:15.895874 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.895790    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:15.895874 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.895827    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:15.956453 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.956427    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:15.994097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.994061    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" event={"ID":"5eb08489-1221-4ab8-9b10-df2003f17b23","Type":"ContainerStarted","Data":"3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a"}
Oct 30 13:13:15.994282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.994121    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" event={"ID":"5eb08489-1221-4ab8-9b10-df2003f17b23","Type":"ContainerStarted","Data":"3c2dcfeb56f974febfd4526eadccc941af807d2a8093224695f295ea7a0759ab"}
Oct 30 13:13:15.997283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.997254    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv" event={"ID":"95db4022-c4ff-4a65-9142-d336e04cb839","Type":"ContainerStarted","Data":"9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0"}
Oct 30 13:13:15.997426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.997316    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:15.997426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:15.997339    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv" event={"ID":"95db4022-c4ff-4a65-9142-d336e04cb839","Type":"ContainerStarted","Data":"51cc1640cb9b95d028eca6ecdb13788d6973bb9c4ff6e40e4902782777b363e1"}
Oct 30 13:13:16.085920 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:16.085874    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podStartSLOduration=1.085859826 podStartE2EDuration="1.085859826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:16.085835923 +0000 UTC m=+933.383053465" watchObservedRunningTime="2026-10-30 13:13:16.085859826 +0000 UTC m=+933.383077369"
Oct 30 13:13:16.151315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:16.151204    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv" podStartSLOduration=2.151188217 podStartE2EDuration="2.151188217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:16.150052223 +0000 UTC m=+933.447269782" watchObservedRunningTime="2026-10-30 13:13:16.151188217 +0000 UTC m=+933.448405760"
Oct 30 13:13:16.295468 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:16.295427    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"]
Oct 30 13:13:16.300268 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:16.300241    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f56a9dd_efa5_4f21_b619_b6bafd9ae785.slice/crio-8c6a792558f0d9b190f7e3b063746cc0fafe1178462110e5802779cc09957659 WatchSource:0}: Error finding container 8c6a792558f0d9b190f7e3b063746cc0fafe1178462110e5802779cc09957659: Status 404 returned error can't find the container with id 8c6a792558f0d9b190f7e3b063746cc0fafe1178462110e5802779cc09957659
Oct 30 13:13:16.553004 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:16.552970    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:16.558528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:16.558505    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:16.741596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:16.741565    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:16.975529 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:16.975469    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:17.005596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:17.005560    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7" event={"ID":"7f56a9dd-efa5-4f21-b619-b6bafd9ae785","Type":"ContainerStarted","Data":"759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7"}
Oct 30 13:13:17.005782 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:17.005663    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7" event={"ID":"7f56a9dd-efa5-4f21-b619-b6bafd9ae785","Type":"ContainerStarted","Data":"8c6a792558f0d9b190f7e3b063746cc0fafe1178462110e5802779cc09957659"}
Oct 30 13:13:17.006611 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:17.006277    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:17.006611 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:17.006363    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:17.007127 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:17.007104    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:17.056628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:17.056583    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7" podStartSLOduration=2.056569439 podStartE2EDuration="2.056569439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:17.055728383 +0000 UTC m=+934.352945925" watchObservedRunningTime="2026-10-30 13:13:17.056569439 +0000 UTC m=+934.353786983"
Oct 30 13:13:17.234677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:17.234589    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:18.447879 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.447848    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:18.583895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.583853    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"]
Oct 30 13:13:18.588004 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.587988    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:18.616184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.616160    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-05942d\""
Oct 30 13:13:18.616184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.616171    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-05942d-dockercfg-2h8s9\""
Oct 30 13:13:18.618719 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.618693    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"]
Oct 30 13:13:18.668252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.668221    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c951714-8bf1-4df2-af09-2c33656636d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz\" (UID: \"7c951714-8bf1-4df2-af09-2c33656636d6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:18.668408 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.668364    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxl2w\" (UniqueName: \"kubernetes.io/projected/7c951714-8bf1-4df2-af09-2c33656636d6-kube-api-access-dxl2w\") pod \"payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz\" (UID: \"7c951714-8bf1-4df2-af09-2c33656636d6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:18.716923 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.716578    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:18.769355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.769319    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxl2w\" (UniqueName: \"kubernetes.io/projected/7c951714-8bf1-4df2-af09-2c33656636d6-kube-api-access-dxl2w\") pod \"payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz\" (UID: \"7c951714-8bf1-4df2-af09-2c33656636d6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:18.769617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.769459    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c951714-8bf1-4df2-af09-2c33656636d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz\" (UID: \"7c951714-8bf1-4df2-af09-2c33656636d6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:18.770301 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.770263    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c951714-8bf1-4df2-af09-2c33656636d6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz\" (UID: \"7c951714-8bf1-4df2-af09-2c33656636d6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:18.814946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.814908    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxl2w\" (UniqueName: \"kubernetes.io/projected/7c951714-8bf1-4df2-af09-2c33656636d6-kube-api-access-dxl2w\") pod \"payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz\" (UID: \"7c951714-8bf1-4df2-af09-2c33656636d6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:18.902493 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.902453    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:18.963769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:18.963735    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:19.248507 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:19.248421    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"]
Oct 30 13:13:19.253310 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:19.253265    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c951714_8bf1_4df2_af09_2c33656636d6.slice/crio-1eef3844ae2a9919aa3016805a3b686d50a771dfdc42c16954ec988aae7a5ece WatchSource:0}: Error finding container 1eef3844ae2a9919aa3016805a3b686d50a771dfdc42c16954ec988aae7a5ece: Status 404 returned error can't find the container with id 1eef3844ae2a9919aa3016805a3b686d50a771dfdc42c16954ec988aae7a5ece
Oct 30 13:13:19.299178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:19.299154    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:19.908156 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:19.908120    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:20.042081 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:20.042000    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz" event={"ID":"7c951714-8bf1-4df2-af09-2c33656636d6","Type":"ContainerStarted","Data":"2edd0f5d4bc95c1b64ea31e3bac23ec3f707ab2c55074287ac411c91134d8494"}
Oct 30 13:13:20.042081 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:20.042049    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz" event={"ID":"7c951714-8bf1-4df2-af09-2c33656636d6","Type":"ContainerStarted","Data":"1eef3844ae2a9919aa3016805a3b686d50a771dfdc42c16954ec988aae7a5ece"}
Oct 30 13:13:20.042318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:20.042133    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:20.099595 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:20.099540    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz" podStartSLOduration=2.099523959 podStartE2EDuration="2.099523959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:20.097004496 +0000 UTC m=+937.394222038" watchObservedRunningTime="2026-10-30 13:13:20.099523959 +0000 UTC m=+937.396741498"
Oct 30 13:13:20.142136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:20.142106    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:21.869019 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:21.868984    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:22.077534 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:22.077487    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:24.485116 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:24.485084    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:24.839547 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:24.839435    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:25.058094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:25.058025    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:25.745111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:25.744779    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:25.996542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:25.996459    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"]
Oct 30 13:13:26.002711 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.002684    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:13:26.002858 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.002735    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:26.003118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.003090    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:13:26.030255 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.030221    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-f8a55e-dockercfg-m9n49\""
Oct 30 13:13:26.044793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.044755    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"]
Oct 30 13:13:26.069847 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.069795    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:26.094325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.094280    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-f8a55e\""
Oct 30 13:13:26.195809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.195777    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx\" (UID: \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:26.196039 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.196012    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8jvj\" (UniqueName: \"kubernetes.io/projected/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-kube-api-access-j8jvj\") pod \"payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx\" (UID: \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:26.297192 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.297106    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8jvj\" (UniqueName: \"kubernetes.io/projected/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-kube-api-access-j8jvj\") pod \"payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx\" (UID: \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:26.297384 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.297245    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx\" (UID: \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:26.297924 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.297902    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx\" (UID: \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:26.364328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.361962    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8jvj\" (UniqueName: \"kubernetes.io/projected/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-kube-api-access-j8jvj\") pod \"payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx\" (UID: \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:26.632835 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.632498    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:26.763811 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.763783    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"]
Oct 30 13:13:26.768030 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.768007    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:26.797697 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.797660    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-05942d-serving-cert\""
Oct 30 13:13:26.797857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.797729    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 13:13:26.797857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.797823    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qrbm\""
Oct 30 13:13:26.801270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.801246    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"]
Oct 30 13:13:26.904608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.904540    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:26.904608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.904597    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-metrics-certs\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:26.904797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.904730    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19cd30f1-725e-4ffa-855e-8a80e928ce86-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:26.904797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.904787    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grwsv\" (UniqueName: \"kubernetes.io/projected/19cd30f1-725e-4ffa-855e-8a80e928ce86-kube-api-access-grwsv\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:26.950198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.950153    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"]
Oct 30 13:13:26.954020 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:26.953995    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6da6ab_fcf2_433e_b12a_dcea9a1daff7.slice/crio-4e79f687ae999c881276c831ba5bbefcd2a680ca910433a078a85ba1dd56e21f WatchSource:0}: Error finding container 4e79f687ae999c881276c831ba5bbefcd2a680ca910433a078a85ba1dd56e21f: Status 404 returned error can't find the container with id 4e79f687ae999c881276c831ba5bbefcd2a680ca910433a078a85ba1dd56e21f
Oct 30 13:13:26.959809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:26.959711    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:27.006513 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.006388    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19cd30f1-725e-4ffa-855e-8a80e928ce86-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:27.006513 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.006457    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grwsv\" (UniqueName: \"kubernetes.io/projected/19cd30f1-725e-4ffa-855e-8a80e928ce86-kube-api-access-grwsv\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:27.006957 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.006539    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:27.006957 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.006587    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-metrics-certs\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:27.007080 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:13:27.007040    5781 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-serving-cert: secret "maas-api-e2e-worker-w1-05942d-serving-cert" not found
Oct 30 13:13:27.007144 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:13:27.007118    5781 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls podName:19cd30f1-725e-4ffa-855e-8a80e928ce86 nodeName:}" failed. No retries permitted until 2026-10-30 13:13:27.507098365 +0000 UTC m=+944.804315900 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls") pod "maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs" (UID: "19cd30f1-725e-4ffa-855e-8a80e928ce86") : secret "maas-api-e2e-worker-w1-05942d-serving-cert" not found
Oct 30 13:13:27.007571 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.007429    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19cd30f1-725e-4ffa-855e-8a80e928ce86-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:27.008830 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.008794    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:13:27.021635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.018888    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-metrics-certs\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:27.046838 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.046804    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grwsv\" (UniqueName: \"kubernetes.io/projected/19cd30f1-725e-4ffa-855e-8a80e928ce86-kube-api-access-grwsv\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:27.086328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.085491    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx" event={"ID":"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7","Type":"ContainerStarted","Data":"09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1"}
Oct 30 13:13:27.086328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.085551    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx" event={"ID":"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7","Type":"ContainerStarted","Data":"4e79f687ae999c881276c831ba5bbefcd2a680ca910433a078a85ba1dd56e21f"}
Oct 30 13:13:27.086328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.085595    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:27.165995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.165919    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:27.343252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.343188    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:27.516315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.512905    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:27.539050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.538963    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls\") pod \"maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:27.683220 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:27.683176    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:28.038871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:28.038763    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx" podStartSLOduration=3.038741959 podStartE2EDuration="3.038741959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:27.137635076 +0000 UTC m=+944.434852620" watchObservedRunningTime="2026-10-30 13:13:28.038741959 +0000 UTC m=+945.335959504"
Oct 30 13:13:28.041024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:28.040994    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"]
Oct 30 13:13:28.043402 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:13:28.043367    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19cd30f1_725e_4ffa_855e_8a80e928ce86.slice/crio-1b5a06f79b746235833944787790643de54caa3ece9081fdcbb25146b3e8f7f5 WatchSource:0}: Error finding container 1b5a06f79b746235833944787790643de54caa3ece9081fdcbb25146b3e8f7f5: Status 404 returned error can't find the container with id 1b5a06f79b746235833944787790643de54caa3ece9081fdcbb25146b3e8f7f5
Oct 30 13:13:28.045187 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:28.045167    5781 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:13:28.092687 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:28.092649    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs" event={"ID":"19cd30f1-725e-4ffa-855e-8a80e928ce86","Type":"ContainerStarted","Data":"1b5a06f79b746235833944787790643de54caa3ece9081fdcbb25146b3e8f7f5"}
Oct 30 13:13:29.041431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:29.040639    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:13:29.067003 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:29.065486    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:30.151260 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.151221    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs" event={"ID":"19cd30f1-725e-4ffa-855e-8a80e928ce86","Type":"ContainerStarted","Data":"1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f"}
Oct 30 13:13:30.151737 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.151316    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:30.153018 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.152991    5781 generic.go:356] "Generic (PLEG): container finished" podID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerID="70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf" exitCode=0
Oct 30 13:13:30.153158 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.153041    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" event={"ID":"519f908a-bf41-4285-8ac4-1abe0a9d893e","Type":"ContainerDied","Data":"70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf"}
Oct 30 13:13:30.153377 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.153361    5781 scope.go:122] "RemoveContainer" containerID="70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf"
Oct 30 13:13:30.154565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.154538    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:30.154656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.154605    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:30.154656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.154637    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:30.193649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.193608    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs" podStartSLOduration=2.269298097 podStartE2EDuration="4.193595952s" totalImagesPullingTime="1.924297855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:13:30.192381645 +0000 UTC m=+947.489599188" watchObservedRunningTime="2026-10-30 13:13:30.193595952 +0000 UTC m=+947.490813494"
Oct 30 13:13:30.610818 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:30.610739    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:31.056310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:31.055468    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:13:31.177187 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:31.177148    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" event={"ID":"519f908a-bf41-4285-8ac4-1abe0a9d893e","Type":"ContainerStarted","Data":"d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779"}
Oct 30 13:13:31.291715 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:31.291678    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:31.504871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:31.504837    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:31.505174 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:31.505045    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 13:13:31.505421 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:31.505395    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 13:13:31.997424 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:13:31.997391    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:13:32.505920 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:32.505883    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 13:13:32.506409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:32.505956    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 13:13:33.376905 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:33.376709    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:33.505722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:33.505671    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 13:13:33.505916 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:33.505734    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 13:13:33.536570 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:33.536452    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:33.971594 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:33.969800    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:34.504787 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:34.504754    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 13:13:34.504950 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:34.504809    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 13:13:34.740325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:34.733769    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:34.948740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:34.948706    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:35.132650 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:35.132595    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:35.505130 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:35.505096    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Oct 30 13:13:35.505352 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:35.505143    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Oct 30 13:13:35.729451 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:35.729366    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:36.011722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.011651    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:36.185921 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.185889    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:13:36.245030 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.244997    5781 generic.go:356] "Generic (PLEG): container finished" podID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerID="96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e" exitCode=0
Oct 30 13:13:36.246345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.245258    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" event={"ID":"e520d651-7215-4f0d-9ecd-c8eace846b1a","Type":"ContainerDied","Data":"96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e"}
Oct 30 13:13:36.246345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.245864    5781 scope.go:122] "RemoveContainer" containerID="96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e"
Oct 30 13:13:36.247565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.247523    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:36.247668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.247639    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:36.247733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.247692    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:36.300316 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.298814    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:36.683490 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.683459    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:36.684764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.684743    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:36.685635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:36.685609    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:13:37.058021 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:37.057940    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:37.259328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:37.258327    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" event={"ID":"e520d651-7215-4f0d-9ecd-c8eace846b1a","Type":"ContainerStarted","Data":"816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1"}
Oct 30 13:13:37.332154 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:37.332029    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:37.344340 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:37.343865    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 13:13:37.344340 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:37.343931    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 13:13:37.944463 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:37.944425    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:38.096664 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:38.096635    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:13:38.125747 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:38.125661    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:38.336360 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:38.335765    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 13:13:38.336360 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:38.335837    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 13:13:38.986459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:38.986420    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:13:38.986641 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:38.986472    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:13:39.338121 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:39.337956    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 13:13:39.338121 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:39.338025    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 13:13:40.331311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:40.330071    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 13:13:40.331311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:40.330130    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 13:13:40.477310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:40.477096    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:41.287805 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:41.287769    5781 generic.go:356] "Generic (PLEG): container finished" podID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerID="49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a" exitCode=0
Oct 30 13:13:41.288055 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:41.287913    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" event={"ID":"cd4e4e0f-5c48-4804-ad64-f9121adb82db","Type":"ContainerDied","Data":"49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a"}
Oct 30 13:13:41.288436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:41.288413    5781 scope.go:122] "RemoveContainer" containerID="49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a"
Oct 30 13:13:41.289811 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:41.289779    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:41.289909 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:41.289850    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:41.289909 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:41.289881    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:41.331363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:41.329452    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 13:13:41.331363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:41.329527    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 13:13:42.337750 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:42.337319    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 13:13:42.337750 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:42.337400    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 13:13:42.348255 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:42.348216    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" event={"ID":"cd4e4e0f-5c48-4804-ad64-f9121adb82db","Type":"ContainerStarted","Data":"1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0"}
Oct 30 13:13:42.825207 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:42.825170    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:43.189142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:43.189112    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:43.189330 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:43.189265    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 13:13:43.189393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:43.189323    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 13:13:43.362313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:43.361872    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:43.369168 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:43.368899    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:13:44.189800 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:44.189764    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 13:13:44.189982 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:44.189813    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 13:13:45.189734 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:45.189703    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 13:13:45.190199 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:45.189763    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 13:13:45.232214 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:45.232082    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:45.466895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:45.466812    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:46.089781 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:46.089747    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:46.189485 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:46.189438    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 13:13:46.189674 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:46.189510    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 13:13:46.318743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:46.318691    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:47.188977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:47.188937    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 13:13:47.189155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:47.189007    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 13:13:47.289101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:47.289053    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:47.896252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:47.896193    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:48.047498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:48.047449    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:13:48.047690 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:48.047532    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:13:48.194586 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:48.194553    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:48.195015 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:48.194968    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:48.195732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:48.195710    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:13:49.413982 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:49.413893    5781 generic.go:356] "Generic (PLEG): container finished" podID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerID="3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a" exitCode=0
Oct 30 13:13:49.414440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:49.413990    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" event={"ID":"5eb08489-1221-4ab8-9b10-df2003f17b23","Type":"ContainerDied","Data":"3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a"}
Oct 30 13:13:49.414506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:49.414471    5781 scope.go:122] "RemoveContainer" containerID="3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a"
Oct 30 13:13:49.415812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:49.415784    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:49.415872 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:49.415849    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:49.415907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:49.415884    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:13:50.094898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:50.094865    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:50.443490 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:50.443455    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" event={"ID":"5eb08489-1221-4ab8-9b10-df2003f17b23","Type":"ContainerStarted","Data":"04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646"}
Oct 30 13:13:50.538951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:50.538915    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:50.553036 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:50.553006    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:50.553244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:50.553218    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 13:13:50.553342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:50.553276    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 13:13:51.125176 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:51.125103    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:51.374757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:51.374724    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:51.553088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:51.553050    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 13:13:51.553556 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:51.553101    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 13:13:52.553042 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:52.553006    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 13:13:52.553217 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:52.553064    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 13:13:52.638740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:52.638706    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:52.987636 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:52.987597    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:53.553796 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:53.553760    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 13:13:53.554280 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:53.553824    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 13:13:54.537374 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:54.533444    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:54.553690 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:54.553655    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 13:13:54.553857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:54.553705    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 13:13:54.733275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:54.733239    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:55.777404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:55.777372    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:55.777908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:55.777802    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:55.779190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:55.779156    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:13:56.711647 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:56.711571    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:58.688097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:58.688036    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:13:59.479298 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:13:59.479246    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:00.189935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:00.189898    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:00.366704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:00.366642    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:00.546404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:00.546311    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:00.847282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:00.847177    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:01.075802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:01.075756    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:01.340800 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:01.340689    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:01.588577 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:01.588517    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:01.996023 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:14:01.995991    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:14:03.082846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:03.082766    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:04.881574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:04.881536    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:06.478209 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:06.478170    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:07.421577 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.421544    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"]
Oct 30 13:14:07.426155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.426125    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.444844 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.444818    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"openshift-service-ca.crt\""
Oct 30 13:14:07.451934 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.451909    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"kube-root-ca.crt\""
Oct 30 13:14:07.452056 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.452008    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"default-dockercfg-vp8gb\""
Oct 30 13:14:07.452122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.452056    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"simulator-w1-05942d-kserve-self-signed-certs\""
Oct 30 13:14:07.458767 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.458742    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"]
Oct 30 13:14:07.531827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.531797    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:07.539552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.539526    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ab619b4-ab20-4655-8def-7ed825afab2f-tls-certs\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.539740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.539603    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-dshm\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.539740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.539704    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqdqn\" (UniqueName: \"kubernetes.io/projected/0ab619b4-ab20-4655-8def-7ed825afab2f-kube-api-access-gqdqn\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.539870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.539751    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-home\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.539870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.539771    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-tmp-dir\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.539870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.539848    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-model-cache\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.640853 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.640817    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-dshm\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.641053 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.640880    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqdqn\" (UniqueName: \"kubernetes.io/projected/0ab619b4-ab20-4655-8def-7ed825afab2f-kube-api-access-gqdqn\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.641053 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.641035    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-home\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.641172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.641070    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-tmp-dir\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.641172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.641097    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-model-cache\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.641172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.641168    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ab619b4-ab20-4655-8def-7ed825afab2f-tls-certs\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.641430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.641409    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-home\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.641499 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.641474    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-tmp-dir\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.641568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.641500    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-model-cache\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.643266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.643240    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-dshm\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.643513 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.643497    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ab619b4-ab20-4655-8def-7ed825afab2f-tls-certs\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.663470 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.663444    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqdqn\" (UniqueName: \"kubernetes.io/projected/0ab619b4-ab20-4655-8def-7ed825afab2f-kube-api-access-gqdqn\") pod \"simulator-w1-05942d-kserve-dc9f8bdcb-64dxg\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") " pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:07.738941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:07.738862    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:08.044564 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:08.044466    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"]
Oct 30 13:14:08.048760 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:14:08.048729    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ab619b4_ab20_4655_8def_7ed825afab2f.slice/crio-ef929ede46d9c7b3dd32d02705a462c8e4efc12346f16336caafaf267fca803f WatchSource:0}: Error finding container ef929ede46d9c7b3dd32d02705a462c8e4efc12346f16336caafaf267fca803f: Status 404 returned error can't find the container with id ef929ede46d9c7b3dd32d02705a462c8e4efc12346f16336caafaf267fca803f
Oct 30 13:14:08.247701 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:08.247666    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:08.545819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:08.545788    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" event={"ID":"0ab619b4-ab20-4655-8def-7ed825afab2f","Type":"ContainerStarted","Data":"13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737"}
Oct 30 13:14:08.545819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:08.545827    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" event={"ID":"0ab619b4-ab20-4655-8def-7ed825afab2f","Type":"ContainerStarted","Data":"ef929ede46d9c7b3dd32d02705a462c8e4efc12346f16336caafaf267fca803f"}
Oct 30 13:14:08.589302 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:08.589235    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podStartSLOduration=1.589221753 podStartE2EDuration="1.589221753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:08.588275364 +0000 UTC m=+985.885492905" watchObservedRunningTime="2026-10-30 13:14:08.589221753 +0000 UTC m=+985.886439314"
Oct 30 13:14:08.740028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:08.739987    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:09.150953 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:09.150914    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:10.104574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:10.104546    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:11.949691 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:11.949656    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:12.381775 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.381694    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"]
Oct 30 13:14:12.385923 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.385896    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.411746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.411703    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"default-dockercfg-6fxwq\""
Oct 30 13:14:12.411746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.411703    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"simulator-w2-f76c09-kserve-self-signed-certs\""
Oct 30 13:14:12.411937 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.411759    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"openshift-service-ca.crt\""
Oct 30 13:14:12.411937 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.411777    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"kube-root-ca.crt\""
Oct 30 13:14:12.414583 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.414562    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"]
Oct 30 13:14:12.481578 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.481549    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:12.496835 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.496806    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-model-cache\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.497031 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.496853    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f5098eb-9409-4853-a343-8beeec46ab48-tls-certs\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.497031 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.496874    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-tmp-dir\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.497031 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.497013    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-dshm\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.497178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.497057    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-home\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.497233 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.497203    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls8kg\" (UniqueName: \"kubernetes.io/projected/1f5098eb-9409-4853-a343-8beeec46ab48-kube-api-access-ls8kg\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.597989 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.597955    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-dshm\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.598175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.598002    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-home\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.598175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.598046    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls8kg\" (UniqueName: \"kubernetes.io/projected/1f5098eb-9409-4853-a343-8beeec46ab48-kube-api-access-ls8kg\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.598175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.598080    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-model-cache\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.598175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.598113    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f5098eb-9409-4853-a343-8beeec46ab48-tls-certs\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.598175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.598132    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-tmp-dir\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.598552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.598514    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-home\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.598628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.598556    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-model-cache\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.598628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.598602    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-tmp-dir\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.600148 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.600125    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-dshm\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.600499 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.600483    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f5098eb-9409-4853-a343-8beeec46ab48-tls-certs\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.641217 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.641123    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls8kg\" (UniqueName: \"kubernetes.io/projected/1f5098eb-9409-4853-a343-8beeec46ab48-kube-api-access-ls8kg\") pod \"simulator-w2-f76c09-kserve-bf97b8948-gdscm\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") " pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:12.698318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:12.698254    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:13.005210 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:13.005174    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"]
Oct 30 13:14:13.010897 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:14:13.010871    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f5098eb_9409_4853_a343_8beeec46ab48.slice/crio-5009de971f3246ca2031d882c7d2068c8f7c8d2ae17e19caea16234b64729327 WatchSource:0}: Error finding container 5009de971f3246ca2031d882c7d2068c8f7c8d2ae17e19caea16234b64729327: Status 404 returned error can't find the container with id 5009de971f3246ca2031d882c7d2068c8f7c8d2ae17e19caea16234b64729327
Oct 30 13:14:13.574156 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:13.574116    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" event={"ID":"1f5098eb-9409-4853-a343-8beeec46ab48","Type":"ContainerStarted","Data":"5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952"}
Oct 30 13:14:13.574386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:13.574165    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" event={"ID":"1f5098eb-9409-4853-a343-8beeec46ab48","Type":"ContainerStarted","Data":"5009de971f3246ca2031d882c7d2068c8f7c8d2ae17e19caea16234b64729327"}
Oct 30 13:14:13.617466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:13.617419    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podStartSLOduration=1.617402394 podStartE2EDuration="1.617402394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:13.616446662 +0000 UTC m=+990.913664206" watchObservedRunningTime="2026-10-30 13:14:13.617402394 +0000 UTC m=+990.914619975"
Oct 30 13:14:13.698420 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:13.698382    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:13.747616 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:13.747585    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:15.647790 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:15.647753    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:17.728991 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:17.728956    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:17.742589 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:17.741694    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:17.760749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:17.760716    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:18.612806 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:18.612777    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:14:19.507831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:19.507794    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:21.100535 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.100499    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"]
Oct 30 13:14:21.106127 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.106100    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.133384 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.133360    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-f8a55e\"/\"kube-root-ca.crt\""
Oct 30 13:14:21.133762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.133730    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-f8a55e\"/\"openshift-service-ca.crt\""
Oct 30 13:14:21.133934 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.133741    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f8a55e\"/\"default-dockercfg-g4m6s\""
Oct 30 13:14:21.133934 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.133802    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f8a55e\"/\"simulator-w5-f8a55e-kserve-self-signed-certs\""
Oct 30 13:14:21.137334 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.137313    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"]
Oct 30 13:14:21.143552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.143525    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:21.195588 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.195555    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tmp-dir\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.195737 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.195613    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhbxl\" (UniqueName: \"kubernetes.io/projected/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-kube-api-access-dhbxl\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.195737 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.195718    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-home\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.195855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.195836    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tls-certs\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.195897 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.195873    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-dshm\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.195936 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.195896    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-model-cache\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.297083 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.297052    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-model-cache\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.297266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.297125    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tmp-dir\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.297266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.297162    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhbxl\" (UniqueName: \"kubernetes.io/projected/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-kube-api-access-dhbxl\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.297266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.297202    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-home\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.297467 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.297282    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tls-certs\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.297467 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.297369    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-dshm\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.297568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.297494    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-model-cache\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.297568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.297535    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tmp-dir\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.297635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.297584    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-home\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.299560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.299537    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-dshm\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.299786 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.299766    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tls-certs\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.328843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.328801    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhbxl\" (UniqueName: \"kubernetes.io/projected/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-kube-api-access-dhbxl\") pod \"simulator-w5-f8a55e-kserve-564878cb54-tvgk8\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") " pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.342050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.342012    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:21.422480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.421820    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:21.764658 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:21.764551    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"]
Oct 30 13:14:21.768610 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:14:21.768586    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9685e2b_61e0_4b77_9d7f_fae0e89e5d0b.slice/crio-9e003aeb1b5cd4d69a322e23deea99ba3dfce0c9b267c7be72d846d7e49fcb60 WatchSource:0}: Error finding container 9e003aeb1b5cd4d69a322e23deea99ba3dfce0c9b267c7be72d846d7e49fcb60: Status 404 returned error can't find the container with id 9e003aeb1b5cd4d69a322e23deea99ba3dfce0c9b267c7be72d846d7e49fcb60
Oct 30 13:14:22.624182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:22.624142    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" event={"ID":"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b","Type":"ContainerStarted","Data":"13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c"}
Oct 30 13:14:22.624182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:22.624182    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" event={"ID":"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b","Type":"ContainerStarted","Data":"9e003aeb1b5cd4d69a322e23deea99ba3dfce0c9b267c7be72d846d7e49fcb60"}
Oct 30 13:14:22.665605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:22.665552    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podStartSLOduration=1.665539083 podStartE2EDuration="1.665539083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:22.663495793 +0000 UTC m=+999.960713336" watchObservedRunningTime="2026-10-30 13:14:22.665539083 +0000 UTC m=+999.962756624"
Oct 30 13:14:22.698390 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:22.698361    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:22.711233 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:22.711208    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:22.884575 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:22.884497    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:23.423266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:23.423234    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:23.641014 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:23.640985    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:14:23.779636 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:23.779562    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:24.694975 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:24.694935    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:26.422954 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:26.422919    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:27.007251 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:27.007220    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:28.599950 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:28.599915    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:29.481467 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:29.481432    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:30.785037 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:30.784996    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:30.959967 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:30.959936    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:31.423912 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:31.423880    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:31.436055 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:31.436033    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:31.687120 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:31.687050    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:14:31.995135 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:14:31.995058    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:14:32.709814 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:32.709783    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:34.587751 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:34.587719    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:36.621730 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:36.621703    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:36.931403 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:36.931371    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:38.401035 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:38.400966    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:40.193790 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:40.193710    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:41.933682 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:41.933645    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:43.686744 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:43.686697    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:45.086792 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.086760    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"]
Oct 30 13:14:45.092067 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.092045    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.117749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.117725    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-05942d\"/\"premium-w1-05942d-kserve-self-signed-certs\""
Oct 30 13:14:45.120632 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.120612    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"]
Oct 30 13:14:45.173860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.173823    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:45.182050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.182028    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-dshm\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.182141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.182067    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tmp-dir\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.182141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.182094    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4njjc\" (UniqueName: \"kubernetes.io/projected/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-kube-api-access-4njjc\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.182235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.182216    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-model-cache\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.182382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.182367    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tls-certs\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.182428 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.182411    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-home\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.284387 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.284348    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-model-cache\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.284594 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.284496    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tls-certs\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.284594 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.284554    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-home\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.284712 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.284616    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-dshm\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.284712 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.284652    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tmp-dir\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.284712 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.284682    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4njjc\" (UniqueName: \"kubernetes.io/projected/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-kube-api-access-4njjc\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.285552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.285522    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-model-cache\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.286259 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.286235    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tmp-dir\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.286422 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.286327    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-home\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.288727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.288242    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-dshm\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.288727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.288526    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tls-certs\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.307825 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.307785    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4njjc\" (UniqueName: \"kubernetes.io/projected/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-kube-api-access-4njjc\") pod \"premium-w1-05942d-kserve-56c45f49bf-f6ttg\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") " pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.404486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.404408    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:45.532253 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.532207    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:45.724739 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.724706    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"]
Oct 30 13:14:45.729636 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:14:45.729608    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod520ca5aa_6058_4d3b_8ab0_75e6fdf6fb99.slice/crio-998c680aaa6f15610955cf235241c31a97ff0b0c470674177fbf326a16a92736 WatchSource:0}: Error finding container 998c680aaa6f15610955cf235241c31a97ff0b0c470674177fbf326a16a92736: Status 404 returned error can't find the container with id 998c680aaa6f15610955cf235241c31a97ff0b0c470674177fbf326a16a92736
Oct 30 13:14:45.755092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:45.755061    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" event={"ID":"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99","Type":"ContainerStarted","Data":"998c680aaa6f15610955cf235241c31a97ff0b0c470674177fbf326a16a92736"}
Oct 30 13:14:46.763000 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:46.762956    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" event={"ID":"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99","Type":"ContainerStarted","Data":"716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7"}
Oct 30 13:14:46.831707 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:46.831653    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podStartSLOduration=1.831638781 podStartE2EDuration="1.831638781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:46.828452296 +0000 UTC m=+1024.125669849" watchObservedRunningTime="2026-10-30 13:14:46.831638781 +0000 UTC m=+1024.128856355"
Oct 30 13:14:47.404975 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:47.404937    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:47.751252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:47.751226    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:49.433365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:49.426416    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:51.287897 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.286304    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:51.600977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.600902    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"]
Oct 30 13:14:51.606884 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.606859    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.621834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.621811    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-88f6a4-openshift-default-dockercfg-tgcfz\""
Oct 30 13:14:51.661376 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.661342    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"]
Oct 30 13:14:51.785992 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.785961    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.786205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.786020    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.786205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.786198    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.786331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.786266    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.786448 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.786419    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khkmx\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-kube-api-access-khkmx\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.786511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.786468    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.786572 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.786510    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.786619 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.786569    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.786619 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.786600    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c14696d-3c86-4d10-93cb-bee727caa66d-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.887416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.887326    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.887416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.887378    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.887416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.887411    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c14696d-3c86-4d10-93cb-bee727caa66d-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.887699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.887448    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.887699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.887477    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.887699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.887560    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.887699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.887626    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.887699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.887677    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khkmx\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-kube-api-access-khkmx\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.890365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.889764    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.890365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.889806    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.891557 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.890844    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.896262 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.894486    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.896262 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.894545    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.896262 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.895420    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c14696d-3c86-4d10-93cb-bee727caa66d-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.899774 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.899749    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.901709 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.901688    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.956414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.955423    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khkmx\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-kube-api-access-khkmx\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:51.960308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:51.956818    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:52.224578 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:52.224544    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:52.576939 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:52.576913    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"]
Oct 30 13:14:52.579640 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:14:52.579609    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c14696d_3c86_4d10_93cb_bee727caa66d.slice/crio-ad6010048228e858c085217f536eac9169faf4529429f431700c36de91730dd6 WatchSource:0}: Error finding container ad6010048228e858c085217f536eac9169faf4529429f431700c36de91730dd6: Status 404 returned error can't find the container with id ad6010048228e858c085217f536eac9169faf4529429f431700c36de91730dd6
Oct 30 13:14:52.582147 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:52.582107    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:14:52.582226 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:52.582200    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:14:52.582271 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:52.582245    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:14:52.804560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:52.804519    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" event={"ID":"2c14696d-3c86-4d10-93cb-bee727caa66d","Type":"ContainerStarted","Data":"0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53"}
Oct 30 13:14:52.804748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:52.804588    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" event={"ID":"2c14696d-3c86-4d10-93cb-bee727caa66d","Type":"ContainerStarted","Data":"ad6010048228e858c085217f536eac9169faf4529429f431700c36de91730dd6"}
Oct 30 13:14:52.867606 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:52.866837    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" podStartSLOduration=1.866817518 podStartE2EDuration="1.866817518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:52.86299642 +0000 UTC m=+1030.160213985" watchObservedRunningTime="2026-10-30 13:14:52.866817518 +0000 UTC m=+1030.164035061"
Oct 30 13:14:53.110968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:53.110901    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:53.226527 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:53.226424    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:53.228708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:53.228680    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:14:53.229002 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:53.228960    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 13:14:54.233688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:54.233649    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:14:54.234268 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:54.234237    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 13:14:54.804830 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:54.804789    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:55.233315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:55.231667    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:14:55.233315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:55.231731    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 13:14:55.405038 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:55.405002    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:55.443215 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:55.443179    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:55.851944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:55.851913    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:14:56.227682 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:56.227646    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:14:56.228000 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:56.227969    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 13:14:56.708792 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:56.708747    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:57.097813 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.093876    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"]
Oct 30 13:14:57.100452 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.100425    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"]
Oct 30 13:14:57.100711 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.100696    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.165366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.165280    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"premium-w2-f76c09-kserve-self-signed-certs\""
Oct 30 13:14:57.167104 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.167067    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:57.195044 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.195013    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tmp-dir\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.195184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.195136    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-model-cache\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.195184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.195175    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p6dc\" (UniqueName: \"kubernetes.io/projected/d4f7d4f7-9ba6-4782-9498-902b09b910f8-kube-api-access-5p6dc\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.195260 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.195233    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-home\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.195317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.195272    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tls-certs\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.195385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.195373    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-dshm\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.245480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.245444    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:14:57.245811 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.245779    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 13:14:57.296730 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.296700    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-model-cache\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.296948 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.296935    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p6dc\" (UniqueName: \"kubernetes.io/projected/d4f7d4f7-9ba6-4782-9498-902b09b910f8-kube-api-access-5p6dc\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.297021 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.297013    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-home\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.297099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.297089    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tls-certs\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.297160 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.297060    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-model-cache\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.297352 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.297203    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-home\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.297441 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.297258    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-dshm\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.297506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.297435    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tmp-dir\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.297717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.297699    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tmp-dir\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.299388 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.299372    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-dshm\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.299640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.299624    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tls-certs\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.348263 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.347973    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p6dc\" (UniqueName: \"kubernetes.io/projected/d4f7d4f7-9ba6-4782-9498-902b09b910f8-kube-api-access-5p6dc\") pod \"premium-w2-f76c09-kserve-76c557b99c-th2xb\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") " pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.472929 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.472894    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:57.807176 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.807143    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"]
Oct 30 13:14:57.810876 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:14:57.810851    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4f7d4f7_9ba6_4782_9498_902b09b910f8.slice/crio-a2a004e6a7ca65dcc2cd4b0c1d182e897b372b6776aab8d68b175a35f8ffe7d5 WatchSource:0}: Error finding container a2a004e6a7ca65dcc2cd4b0c1d182e897b372b6776aab8d68b175a35f8ffe7d5: Status 404 returned error can't find the container with id a2a004e6a7ca65dcc2cd4b0c1d182e897b372b6776aab8d68b175a35f8ffe7d5
Oct 30 13:14:57.884229 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:57.884195    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" event={"ID":"d4f7d4f7-9ba6-4782-9498-902b09b910f8","Type":"ContainerStarted","Data":"a2a004e6a7ca65dcc2cd4b0c1d182e897b372b6776aab8d68b175a35f8ffe7d5"}
Oct 30 13:14:58.226097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:58.225669    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 13:14:58.226097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:58.225729    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 13:14:58.459204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:58.459170    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:14:58.892162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:58.892115    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" event={"ID":"d4f7d4f7-9ba6-4782-9498-902b09b910f8","Type":"ContainerStarted","Data":"fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb"}
Oct 30 13:14:58.936415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:58.936363    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podStartSLOduration=2.936347228 podStartE2EDuration="2.936347228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:58.934127523 +0000 UTC m=+1036.231345066" watchObservedRunningTime="2026-10-30 13:14:58.936347228 +0000 UTC m=+1036.233564771"
Oct 30 13:14:59.473583 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:59.473554    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:14:59.964275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:59.964243    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:59.964773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:59.964735    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:14:59.965602 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:14:59.965581    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:15:00.241133 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.241046    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"]
Oct 30 13:15:00.245110 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.245085    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"]
Oct 30 13:15:00.245242 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.245201    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:00.249162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.249143    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:00.272886 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.272865    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:00.276359 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.276337    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"]
Oct 30 13:15:00.283629 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.281583    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"]
Oct 30 13:15:00.283629 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.281687    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:00.284673 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.284650    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"]
Oct 30 13:15:00.313766 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.313735    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:00.316054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.316032    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"]
Oct 30 13:15:00.319968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.319950    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:00.326002 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.325486    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"]
Oct 30 13:15:00.340322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.335097    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl82x\" (UniqueName: \"kubernetes.io/projected/e3425d5b-5541-40da-8bda-2c099588163e-kube-api-access-bl82x\") pod \"maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd\" (UID: \"e3425d5b-5541-40da-8bda-2c099588163e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:00.340322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.335196    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2dd4b1d-794e-485c-8513-0db94b048a6c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8\" (UID: \"a2dd4b1d-794e-485c-8513-0db94b048a6c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:00.340322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.335335    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a20054-fa55-4bd5-8871-42dfab7a611b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q\" (UID: \"67a20054-fa55-4bd5-8871-42dfab7a611b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:00.340322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.335427    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3425d5b-5541-40da-8bda-2c099588163e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd\" (UID: \"e3425d5b-5541-40da-8bda-2c099588163e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:00.340322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.335559    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxklf\" (UniqueName: \"kubernetes.io/projected/67a20054-fa55-4bd5-8871-42dfab7a611b-kube-api-access-qxklf\") pod \"maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q\" (UID: \"67a20054-fa55-4bd5-8871-42dfab7a611b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:00.340322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.335609    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4qpg\" (UniqueName: \"kubernetes.io/projected/a2dd4b1d-794e-485c-8513-0db94b048a6c-kube-api-access-r4qpg\") pod \"maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8\" (UID: \"a2dd4b1d-794e-485c-8513-0db94b048a6c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:00.346433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.345830    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"]
Oct 30 13:15:00.437142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437108    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a20054-fa55-4bd5-8871-42dfab7a611b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q\" (UID: \"67a20054-fa55-4bd5-8871-42dfab7a611b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:00.437309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437163    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3425d5b-5541-40da-8bda-2c099588163e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd\" (UID: \"e3425d5b-5541-40da-8bda-2c099588163e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:00.437309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437191    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj7j4\" (UniqueName: \"kubernetes.io/projected/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-kube-api-access-pj7j4\") pod \"maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp\" (UID: \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:00.437309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437248    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxklf\" (UniqueName: \"kubernetes.io/projected/67a20054-fa55-4bd5-8871-42dfab7a611b-kube-api-access-qxklf\") pod \"maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q\" (UID: \"67a20054-fa55-4bd5-8871-42dfab7a611b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:00.437309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437271    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4qpg\" (UniqueName: \"kubernetes.io/projected/a2dd4b1d-794e-485c-8513-0db94b048a6c-kube-api-access-r4qpg\") pod \"maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8\" (UID: \"a2dd4b1d-794e-485c-8513-0db94b048a6c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:00.437544 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437368    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp\" (UID: \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:00.437544 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437436    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl82x\" (UniqueName: \"kubernetes.io/projected/e3425d5b-5541-40da-8bda-2c099588163e-kube-api-access-bl82x\") pod \"maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd\" (UID: \"e3425d5b-5541-40da-8bda-2c099588163e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:00.437544 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437503    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2dd4b1d-794e-485c-8513-0db94b048a6c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8\" (UID: \"a2dd4b1d-794e-485c-8513-0db94b048a6c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:00.437973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437948    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a20054-fa55-4bd5-8871-42dfab7a611b-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q\" (UID: \"67a20054-fa55-4bd5-8871-42dfab7a611b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:00.438014 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.437967    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3425d5b-5541-40da-8bda-2c099588163e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd\" (UID: \"e3425d5b-5541-40da-8bda-2c099588163e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:00.438111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.438096    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2dd4b1d-794e-485c-8513-0db94b048a6c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8\" (UID: \"a2dd4b1d-794e-485c-8513-0db94b048a6c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:00.465749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.465719    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4qpg\" (UniqueName: \"kubernetes.io/projected/a2dd4b1d-794e-485c-8513-0db94b048a6c-kube-api-access-r4qpg\") pod \"maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8\" (UID: \"a2dd4b1d-794e-485c-8513-0db94b048a6c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:00.466656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.466637    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl82x\" (UniqueName: \"kubernetes.io/projected/e3425d5b-5541-40da-8bda-2c099588163e-kube-api-access-bl82x\") pod \"maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd\" (UID: \"e3425d5b-5541-40da-8bda-2c099588163e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:00.470609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.470591    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxklf\" (UniqueName: \"kubernetes.io/projected/67a20054-fa55-4bd5-8871-42dfab7a611b-kube-api-access-qxklf\") pod \"maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q\" (UID: \"67a20054-fa55-4bd5-8871-42dfab7a611b\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:00.538679 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.538611    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp\" (UID: \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:00.538840 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.538821    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj7j4\" (UniqueName: \"kubernetes.io/projected/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-kube-api-access-pj7j4\") pod \"maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp\" (UID: \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:00.539234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.539217    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp\" (UID: \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:00.560759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.560737    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:00.566900 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.566878    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:00.571712 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.571689    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj7j4\" (UniqueName: \"kubernetes.io/projected/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-kube-api-access-pj7j4\") pod \"maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp\" (UID: \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:00.599366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.599333    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:00.645935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.645647    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:00.919594 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.919182    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"]
Oct 30 13:15:00.921918 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.921892    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"]
Oct 30 13:15:00.923400 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:00.923377    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a20054_fa55_4bd5_8871_42dfab7a611b.slice/crio-2fa991a81912aa7ea7153513d1bffbb68ddac394327bd7a49bef202aa5890330 WatchSource:0}: Error finding container 2fa991a81912aa7ea7153513d1bffbb68ddac394327bd7a49bef202aa5890330: Status 404 returned error can't find the container with id 2fa991a81912aa7ea7153513d1bffbb68ddac394327bd7a49bef202aa5890330
Oct 30 13:15:00.924182 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:00.924161    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3425d5b_5541_40da_8bda_2c099588163e.slice/crio-41b6c37ba14285962163a0b7c0e291ff7d93383fcfa983ec8c854cdc6c005c88 WatchSource:0}: Error finding container 41b6c37ba14285962163a0b7c0e291ff7d93383fcfa983ec8c854cdc6c005c88: Status 404 returned error can't find the container with id 41b6c37ba14285962163a0b7c0e291ff7d93383fcfa983ec8c854cdc6c005c88
Oct 30 13:15:00.946781 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:00.946749    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"]
Oct 30 13:15:00.947971 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:00.947952    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2dd4b1d_794e_485c_8513_0db94b048a6c.slice/crio-e382b1b9354b75c4195019d022a64eec9808058061e38e31acdbeb55dfe337ea WatchSource:0}: Error finding container e382b1b9354b75c4195019d022a64eec9808058061e38e31acdbeb55dfe337ea: Status 404 returned error can't find the container with id e382b1b9354b75c4195019d022a64eec9808058061e38e31acdbeb55dfe337ea
Oct 30 13:15:01.000894 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:01.000791    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"]
Oct 30 13:15:01.004650 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:01.004627    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb88f0c6b_8e1a_46bf_a024_4e763458e7bc.slice/crio-e0f0ea25bdcc41f0f0148e46ca1fd57fe3bf88d557e958bd80812778462cdc7c WatchSource:0}: Error finding container e0f0ea25bdcc41f0f0148e46ca1fd57fe3bf88d557e958bd80812778462cdc7c: Status 404 returned error can't find the container with id e0f0ea25bdcc41f0f0148e46ca1fd57fe3bf88d557e958bd80812778462cdc7c
Oct 30 13:15:01.916386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:01.916311    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" event={"ID":"67a20054-fa55-4bd5-8871-42dfab7a611b","Type":"ContainerStarted","Data":"2fa991a81912aa7ea7153513d1bffbb68ddac394327bd7a49bef202aa5890330"}
Oct 30 13:15:01.920201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:01.920121    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd" event={"ID":"e3425d5b-5541-40da-8bda-2c099588163e","Type":"ContainerStarted","Data":"41b6c37ba14285962163a0b7c0e291ff7d93383fcfa983ec8c854cdc6c005c88"}
Oct 30 13:15:01.924148 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:01.924097    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" event={"ID":"b88f0c6b-8e1a-46bf-a024-4e763458e7bc","Type":"ContainerStarted","Data":"e0f0ea25bdcc41f0f0148e46ca1fd57fe3bf88d557e958bd80812778462cdc7c"}
Oct 30 13:15:01.927922 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:01.927897    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" event={"ID":"a2dd4b1d-794e-485c-8513-0db94b048a6c","Type":"ContainerStarted","Data":"e382b1b9354b75c4195019d022a64eec9808058061e38e31acdbeb55dfe337ea"}
Oct 30 13:15:01.970864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:01.970790    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:02.021520 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:02.021454    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:15:02.447795 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:02.447759    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:04.237427 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:04.237379    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:04.950025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:04.949995    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" event={"ID":"b88f0c6b-8e1a-46bf-a024-4e763458e7bc","Type":"ContainerStarted","Data":"ac3ebf84b7fa0b143d3b27bd3140534c6fd2883e964ff0526eaa29f1a3bbed84"}
Oct 30 13:15:04.951531 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:04.951507    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" event={"ID":"a2dd4b1d-794e-485c-8513-0db94b048a6c","Type":"ContainerStarted","Data":"9cd7ef7ede6a265084983c724096c2c7c2d367f845ab75806919a57f95b94ed7"}
Oct 30 13:15:04.952920 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:04.952898    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" event={"ID":"67a20054-fa55-4bd5-8871-42dfab7a611b","Type":"ContainerStarted","Data":"e2ce5fcd57ae8b7ffa9863f318f555e392d55ccc4267bd6d5eb4099bb855d115"}
Oct 30 13:15:04.954406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:04.954383    5781 generic.go:356] "Generic (PLEG): container finished" podID="e3425d5b-5541-40da-8bda-2c099588163e" containerID="e41d88a384d5740925548850c51b926cab09acc35f34073324569ccb3e9d5f88" exitCode=60
Oct 30 13:15:04.954488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:04.954458    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd" event={"ID":"e3425d5b-5541-40da-8bda-2c099588163e","Type":"ContainerDied","Data":"e41d88a384d5740925548850c51b926cab09acc35f34073324569ccb3e9d5f88"}
Oct 30 13:15:04.954598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:04.954584    5781 scope.go:122] "RemoveContainer" containerID="e41d88a384d5740925548850c51b926cab09acc35f34073324569ccb3e9d5f88"
Oct 30 13:15:04.989387 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:04.989342    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" podStartSLOduration=1.5980369319999999 podStartE2EDuration="4.989329493s" totalImagesPullingTime="3.391292561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:04.988816775 +0000 UTC m=+1042.286034317" watchObservedRunningTime="2026-10-30 13:15:04.989329493 +0000 UTC m=+1042.286547035"
Oct 30 13:15:05.030235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.030180    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" podStartSLOduration=1.58186443 podStartE2EDuration="5.030166177s" totalImagesPullingTime="3.448301747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:05.029044491 +0000 UTC m=+1042.326262047" watchObservedRunningTime="2026-10-30 13:15:05.030166177 +0000 UTC m=+1042.327383716"
Oct 30 13:15:05.133559 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.133511    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" podStartSLOduration=1.6899813369999999 podStartE2EDuration="5.133497978s" totalImagesPullingTime="3.443516641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:05.13312388 +0000 UTC m=+1042.430341422" watchObservedRunningTime="2026-10-30 13:15:05.133497978 +0000 UTC m=+1042.430715520"
Oct 30 13:15:05.901928 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.901894    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:05.962413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.962384    5781 generic.go:356] "Generic (PLEG): container finished" podID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerID="ac3ebf84b7fa0b143d3b27bd3140534c6fd2883e964ff0526eaa29f1a3bbed84" exitCode=60
Oct 30 13:15:05.962602 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.962421    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" event={"ID":"b88f0c6b-8e1a-46bf-a024-4e763458e7bc","Type":"ContainerDied","Data":"ac3ebf84b7fa0b143d3b27bd3140534c6fd2883e964ff0526eaa29f1a3bbed84"}
Oct 30 13:15:05.962756 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.962739    5781 scope.go:122] "RemoveContainer" containerID="ac3ebf84b7fa0b143d3b27bd3140534c6fd2883e964ff0526eaa29f1a3bbed84"
Oct 30 13:15:05.964004 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.963983    5781 generic.go:356] "Generic (PLEG): container finished" podID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerID="9cd7ef7ede6a265084983c724096c2c7c2d367f845ab75806919a57f95b94ed7" exitCode=60
Oct 30 13:15:05.964122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.964053    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" event={"ID":"a2dd4b1d-794e-485c-8513-0db94b048a6c","Type":"ContainerDied","Data":"9cd7ef7ede6a265084983c724096c2c7c2d367f845ab75806919a57f95b94ed7"}
Oct 30 13:15:05.964364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.964341    5781 scope.go:122] "RemoveContainer" containerID="9cd7ef7ede6a265084983c724096c2c7c2d367f845ab75806919a57f95b94ed7"
Oct 30 13:15:05.965851 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.965786    5781 generic.go:356] "Generic (PLEG): container finished" podID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerID="e2ce5fcd57ae8b7ffa9863f318f555e392d55ccc4267bd6d5eb4099bb855d115" exitCode=60
Oct 30 13:15:05.965941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.965884    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" event={"ID":"67a20054-fa55-4bd5-8871-42dfab7a611b","Type":"ContainerDied","Data":"e2ce5fcd57ae8b7ffa9863f318f555e392d55ccc4267bd6d5eb4099bb855d115"}
Oct 30 13:15:05.966071 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.966058    5781 scope.go:122] "RemoveContainer" containerID="e2ce5fcd57ae8b7ffa9863f318f555e392d55ccc4267bd6d5eb4099bb855d115"
Oct 30 13:15:05.969071 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.969056    5781 generic.go:356] "Generic (PLEG): container finished" podID="e3425d5b-5541-40da-8bda-2c099588163e" containerID="0820af043d539f66390af7c0608dff17a88dfdcd31b719b1e169a18f30428f51" exitCode=60
Oct 30 13:15:05.969126 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.969082    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd" event={"ID":"e3425d5b-5541-40da-8bda-2c099588163e","Type":"ContainerDied","Data":"0820af043d539f66390af7c0608dff17a88dfdcd31b719b1e169a18f30428f51"}
Oct 30 13:15:05.969167 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.969134    5781 scope.go:122] "RemoveContainer" containerID="e41d88a384d5740925548850c51b926cab09acc35f34073324569ccb3e9d5f88"
Oct 30 13:15:05.969423 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:05.969402    5781 scope.go:122] "RemoveContainer" containerID="0820af043d539f66390af7c0608dff17a88dfdcd31b719b1e169a18f30428f51"
Oct 30 13:15:05.969643 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:05.969618    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd_odh-ai-gateway-infra(e3425d5b-5541-40da-8bda-2c099588163e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd" podUID="e3425d5b-5541-40da-8bda-2c099588163e"
Oct 30 13:15:06.976718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.976682    5781 generic.go:356] "Generic (PLEG): container finished" podID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerID="6f109b8e9e814a694d7902e1542ae5a16210ff8f03962398d986658d24dd5ec6" exitCode=60
Oct 30 13:15:06.977122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.976773    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" event={"ID":"b88f0c6b-8e1a-46bf-a024-4e763458e7bc","Type":"ContainerDied","Data":"6f109b8e9e814a694d7902e1542ae5a16210ff8f03962398d986658d24dd5ec6"}
Oct 30 13:15:06.977122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.976847    5781 scope.go:122] "RemoveContainer" containerID="ac3ebf84b7fa0b143d3b27bd3140534c6fd2883e964ff0526eaa29f1a3bbed84"
Oct 30 13:15:06.977122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.977037    5781 scope.go:122] "RemoveContainer" containerID="6f109b8e9e814a694d7902e1542ae5a16210ff8f03962398d986658d24dd5ec6"
Oct 30 13:15:06.977342 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:06.977320    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp_odh-ai-gateway-infra(b88f0c6b-8e1a-46bf-a024-4e763458e7bc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc"
Oct 30 13:15:06.982394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.982371    5781 generic.go:356] "Generic (PLEG): container finished" podID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerID="a6b0a4fbb53d03cac447ce5831311f29b9a1e303ba0cccaafe216cff550fe01b" exitCode=60
Oct 30 13:15:06.982522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.982455    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" event={"ID":"a2dd4b1d-794e-485c-8513-0db94b048a6c","Type":"ContainerDied","Data":"a6b0a4fbb53d03cac447ce5831311f29b9a1e303ba0cccaafe216cff550fe01b"}
Oct 30 13:15:06.982722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.982709    5781 scope.go:122] "RemoveContainer" containerID="a6b0a4fbb53d03cac447ce5831311f29b9a1e303ba0cccaafe216cff550fe01b"
Oct 30 13:15:06.982959 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:06.982942    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8_odh-ai-gateway-infra(a2dd4b1d-794e-485c-8513-0db94b048a6c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c"
Oct 30 13:15:06.984609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.984591    5781 generic.go:356] "Generic (PLEG): container finished" podID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerID="7e343fad712120e5325b451fd5853d00ae191ecf8f7c075287669508f6ef70c3" exitCode=60
Oct 30 13:15:06.984704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.984676    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" event={"ID":"67a20054-fa55-4bd5-8871-42dfab7a611b","Type":"ContainerDied","Data":"7e343fad712120e5325b451fd5853d00ae191ecf8f7c075287669508f6ef70c3"}
Oct 30 13:15:06.984863 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.984850    5781 scope.go:122] "RemoveContainer" containerID="7e343fad712120e5325b451fd5853d00ae191ecf8f7c075287669508f6ef70c3"
Oct 30 13:15:06.985097 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:06.985078    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q_odh-ai-gateway-infra(67a20054-fa55-4bd5-8871-42dfab7a611b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b"
Oct 30 13:15:06.986377 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.986353    5781 scope.go:122] "RemoveContainer" containerID="0820af043d539f66390af7c0608dff17a88dfdcd31b719b1e169a18f30428f51"
Oct 30 13:15:06.986562 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:06.986545    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd_odh-ai-gateway-infra(e3425d5b-5541-40da-8bda-2c099588163e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd" podUID="e3425d5b-5541-40da-8bda-2c099588163e"
Oct 30 13:15:06.993160 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:06.993140    5781 scope.go:122] "RemoveContainer" containerID="9cd7ef7ede6a265084983c724096c2c7c2d367f845ab75806919a57f95b94ed7"
Oct 30 13:15:07.005302 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:07.005274    5781 scope.go:122] "RemoveContainer" containerID="e2ce5fcd57ae8b7ffa9863f318f555e392d55ccc4267bd6d5eb4099bb855d115"
Oct 30 13:15:07.477330 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:07.475799    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:15:07.497164 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:07.497131    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:15:07.777020 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:07.776936    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:07.998846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:07.998810    5781 scope.go:122] "RemoveContainer" containerID="6f109b8e9e814a694d7902e1542ae5a16210ff8f03962398d986658d24dd5ec6"
Oct 30 13:15:07.999375 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:07.999274    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp_odh-ai-gateway-infra(b88f0c6b-8e1a-46bf-a024-4e763458e7bc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc"
Oct 30 13:15:08.000115 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:08.000085    5781 scope.go:122] "RemoveContainer" containerID="a6b0a4fbb53d03cac447ce5831311f29b9a1e303ba0cccaafe216cff550fe01b"
Oct 30 13:15:08.000421 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:08.000393    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8_odh-ai-gateway-infra(a2dd4b1d-794e-485c-8513-0db94b048a6c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c"
Oct 30 13:15:08.002133 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:08.002114    5781 scope.go:122] "RemoveContainer" containerID="7e343fad712120e5325b451fd5853d00ae191ecf8f7c075287669508f6ef70c3"
Oct 30 13:15:08.002373 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:08.002348    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q_odh-ai-gateway-infra(67a20054-fa55-4bd5-8871-42dfab7a611b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b"
Oct 30 13:15:08.020084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:08.020056    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:15:09.744739 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:09.744707    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:11.112678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:11.112640    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:11.478206 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:11.478163    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:13.062578 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:13.062539    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:13.282159 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:13.282127    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:13.520411 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:13.520378    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:14.685366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:14.685312    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:16.206445 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:16.206394    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:16.834872 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:16.834841    5781 scope.go:122] "RemoveContainer" containerID="0820af043d539f66390af7c0608dff17a88dfdcd31b719b1e169a18f30428f51"
Oct 30 13:15:17.374863 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.374774    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"]
Oct 30 13:15:17.379421 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.379399    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.394555 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.394536    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:17.405916 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.405892    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-94af0758-h54vx\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.406048 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.406011    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg8l6\" (UniqueName: \"kubernetes.io/projected/0dfc3095-8c23-49a8-bc62-530830f57e13-kube-api-access-qg8l6\") pod \"maas-api-revoke-sub-94af0758-h54vx\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.406100 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.406065    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-xks-ca\") pod \"maas-api-revoke-sub-94af0758-h54vx\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.410328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.410304    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"]
Oct 30 13:15:17.507022 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.506984    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-94af0758-h54vx\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.507217 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.507068    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg8l6\" (UniqueName: \"kubernetes.io/projected/0dfc3095-8c23-49a8-bc62-530830f57e13-kube-api-access-qg8l6\") pod \"maas-api-revoke-sub-94af0758-h54vx\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.507217 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.507088    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-xks-ca\") pod \"maas-api-revoke-sub-94af0758-h54vx\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.507653 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.507629    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-94af0758-h54vx\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.509486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.509463    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-xks-ca\") pod \"maas-api-revoke-sub-94af0758-h54vx\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.530036 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.530001    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg8l6\" (UniqueName: \"kubernetes.io/projected/0dfc3095-8c23-49a8-bc62-530830f57e13-kube-api-access-qg8l6\") pod \"maas-api-revoke-sub-94af0758-h54vx\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.692735 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.692705    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:17.834195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.834165    5781 scope.go:122] "RemoveContainer" containerID="6f109b8e9e814a694d7902e1542ae5a16210ff8f03962398d986658d24dd5ec6"
Oct 30 13:15:17.899028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.898988    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:17.999429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:17.999405    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"]
Oct 30 13:15:18.002227 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:18.002195    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dfc3095_8c23_49a8_bc62_530830f57e13.slice/crio-45052d735aad71ccd44425cd28244628a6653e2fd57bd932ce1066403342e682 WatchSource:0}: Error finding container 45052d735aad71ccd44425cd28244628a6653e2fd57bd932ce1066403342e682: Status 404 returned error can't find the container with id 45052d735aad71ccd44425cd28244628a6653e2fd57bd932ce1066403342e682
Oct 30 13:15:18.069637 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:18.069601    5781 generic.go:356] "Generic (PLEG): container finished" podID="e3425d5b-5541-40da-8bda-2c099588163e" containerID="084250b9d0a48a5f7f2e1a30b25e22a5056545dafaed16e40c055b327d8b140f" exitCode=60
Oct 30 13:15:18.069804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:18.069671    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd" event={"ID":"e3425d5b-5541-40da-8bda-2c099588163e","Type":"ContainerDied","Data":"084250b9d0a48a5f7f2e1a30b25e22a5056545dafaed16e40c055b327d8b140f"}
Oct 30 13:15:18.069804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:18.069731    5781 scope.go:122] "RemoveContainer" containerID="0820af043d539f66390af7c0608dff17a88dfdcd31b719b1e169a18f30428f51"
Oct 30 13:15:18.070120 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:18.070096    5781 scope.go:122] "RemoveContainer" containerID="084250b9d0a48a5f7f2e1a30b25e22a5056545dafaed16e40c055b327d8b140f"
Oct 30 13:15:18.070465 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:18.070442    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd_odh-ai-gateway-infra(e3425d5b-5541-40da-8bda-2c099588163e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd" podUID="e3425d5b-5541-40da-8bda-2c099588163e"
Oct 30 13:15:18.071098 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:18.071074    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx" event={"ID":"0dfc3095-8c23-49a8-bc62-530830f57e13","Type":"ContainerStarted","Data":"45052d735aad71ccd44425cd28244628a6653e2fd57bd932ce1066403342e682"}
Oct 30 13:15:18.834923 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:18.834896    5781 scope.go:122] "RemoveContainer" containerID="7e343fad712120e5325b451fd5853d00ae191ecf8f7c075287669508f6ef70c3"
Oct 30 13:15:19.081666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.081628    5781 generic.go:356] "Generic (PLEG): container finished" podID="0dfc3095-8c23-49a8-bc62-530830f57e13" containerID="ddffc9dab4ca30c3bf88c46cb5ed78eafa9aa24e0f9a311e936735cbabf42cb1" exitCode=0
Oct 30 13:15:19.081844 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.081708    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx" event={"ID":"0dfc3095-8c23-49a8-bc62-530830f57e13","Type":"ContainerDied","Data":"ddffc9dab4ca30c3bf88c46cb5ed78eafa9aa24e0f9a311e936735cbabf42cb1"}
Oct 30 13:15:19.083726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.083701    5781 generic.go:356] "Generic (PLEG): container finished" podID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerID="9db932ae07ee92ef3a83325f35fc83af1c37e8fc39dc7cccc27e7c8af13995b0" exitCode=60
Oct 30 13:15:19.083859 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.083771    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" event={"ID":"b88f0c6b-8e1a-46bf-a024-4e763458e7bc","Type":"ContainerDied","Data":"9db932ae07ee92ef3a83325f35fc83af1c37e8fc39dc7cccc27e7c8af13995b0"}
Oct 30 13:15:19.083859 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.083821    5781 scope.go:122] "RemoveContainer" containerID="6f109b8e9e814a694d7902e1542ae5a16210ff8f03962398d986658d24dd5ec6"
Oct 30 13:15:19.084182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.084168    5781 scope.go:122] "RemoveContainer" containerID="9db932ae07ee92ef3a83325f35fc83af1c37e8fc39dc7cccc27e7c8af13995b0"
Oct 30 13:15:19.084493 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:19.084473    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp_odh-ai-gateway-infra(b88f0c6b-8e1a-46bf-a024-4e763458e7bc)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc"
Oct 30 13:15:19.122134 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.122110    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"]
Oct 30 13:15:19.275483 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.275458    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:19.329169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.329139    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3425d5b-5541-40da-8bda-2c099588163e-kube-api-access-bl82x\" (UniqueName: \"kubernetes.io/projected/e3425d5b-5541-40da-8bda-2c099588163e-kube-api-access-bl82x\") pod \"e3425d5b-5541-40da-8bda-2c099588163e\" (UID: \"e3425d5b-5541-40da-8bda-2c099588163e\") "
Oct 30 13:15:19.329373 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.329260    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3425d5b-5541-40da-8bda-2c099588163e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3425d5b-5541-40da-8bda-2c099588163e-maas-api-service-ca\") pod \"e3425d5b-5541-40da-8bda-2c099588163e\" (UID: \"e3425d5b-5541-40da-8bda-2c099588163e\") "
Oct 30 13:15:19.329935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.329901    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3425d5b-5541-40da-8bda-2c099588163e-maas-api-service-ca" pod "e3425d5b-5541-40da-8bda-2c099588163e" (UID: "e3425d5b-5541-40da-8bda-2c099588163e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:19.331433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.331404    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3425d5b-5541-40da-8bda-2c099588163e-kube-api-access-bl82x" pod "e3425d5b-5541-40da-8bda-2c099588163e" (UID: "e3425d5b-5541-40da-8bda-2c099588163e"). InnerVolumeSpecName "kube-api-access-bl82x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:19.354592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.354534    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:19.430918 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.430833    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl82x\" (UniqueName: \"kubernetes.io/projected/e3425d5b-5541-40da-8bda-2c099588163e-kube-api-access-bl82x\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:19.430918 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.430862    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3425d5b-5541-40da-8bda-2c099588163e-maas-api-service-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:19.833958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:19.833930    5781 scope.go:122] "RemoveContainer" containerID="a6b0a4fbb53d03cac447ce5831311f29b9a1e303ba0cccaafe216cff550fe01b"
Oct 30 13:15:20.091411 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.091381    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"
Oct 30 13:15:20.091860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.091424    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd" event={"ID":"e3425d5b-5541-40da-8bda-2c099588163e","Type":"ContainerDied","Data":"41b6c37ba14285962163a0b7c0e291ff7d93383fcfa983ec8c854cdc6c005c88"}
Oct 30 13:15:20.091860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.091481    5781 scope.go:122] "RemoveContainer" containerID="084250b9d0a48a5f7f2e1a30b25e22a5056545dafaed16e40c055b327d8b140f"
Oct 30 13:15:20.097685 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.097653    5781 generic.go:356] "Generic (PLEG): container finished" podID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerID="209f42eef21812375b98a09a166775647769eab347f8de4f0f5f41fe63ef3e31" exitCode=60
Oct 30 13:15:20.097830 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.097717    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" event={"ID":"67a20054-fa55-4bd5-8871-42dfab7a611b","Type":"ContainerDied","Data":"209f42eef21812375b98a09a166775647769eab347f8de4f0f5f41fe63ef3e31"}
Oct 30 13:15:20.098283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.098266    5781 scope.go:122] "RemoveContainer" containerID="209f42eef21812375b98a09a166775647769eab347f8de4f0f5f41fe63ef3e31"
Oct 30 13:15:20.098663 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:20.098642    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q_odh-ai-gateway-infra(67a20054-fa55-4bd5-8871-42dfab7a611b)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b"
Oct 30 13:15:20.101966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.101943    5781 scope.go:122] "RemoveContainer" containerID="7e343fad712120e5325b451fd5853d00ae191ecf8f7c075287669508f6ef70c3"
Oct 30 13:15:20.213697 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.213658    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"]
Oct 30 13:15:20.219457 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.219425    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"]
Oct 30 13:15:20.223055 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.223024    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-f8a55e-29889435-ch5cd"]
Oct 30 13:15:20.252088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.252048    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:20.344952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.344924    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-openshift-ca\") pod \"0dfc3095-8c23-49a8-bc62-530830f57e13\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") "
Oct 30 13:15:20.345115 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.344969    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-xks-ca\") pod \"0dfc3095-8c23-49a8-bc62-530830f57e13\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") "
Oct 30 13:15:20.345211 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.345195    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dfc3095-8c23-49a8-bc62-530830f57e13-kube-api-access-qg8l6\" (UniqueName: \"kubernetes.io/projected/0dfc3095-8c23-49a8-bc62-530830f57e13-kube-api-access-qg8l6\") pod \"0dfc3095-8c23-49a8-bc62-530830f57e13\" (UID: \"0dfc3095-8c23-49a8-bc62-530830f57e13\") "
Oct 30 13:15:20.345367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.345336    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-openshift-ca" pod "0dfc3095-8c23-49a8-bc62-530830f57e13" (UID: "0dfc3095-8c23-49a8-bc62-530830f57e13"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:20.346196 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.345786    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:20.347917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.347245    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-xks-ca" pod "0dfc3095-8c23-49a8-bc62-530830f57e13" (UID: "0dfc3095-8c23-49a8-bc62-530830f57e13"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:20.357327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.356041    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dfc3095-8c23-49a8-bc62-530830f57e13-kube-api-access-qg8l6" pod "0dfc3095-8c23-49a8-bc62-530830f57e13" (UID: "0dfc3095-8c23-49a8-bc62-530830f57e13"). InnerVolumeSpecName "kube-api-access-qg8l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:20.367415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.367393    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:20.447140 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.447107    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-kube-api-access-pj7j4\" (UniqueName: \"kubernetes.io/projected/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-kube-api-access-pj7j4\") pod \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\" (UID: \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\") "
Oct 30 13:15:20.447357 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.447343    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-maas-api-service-ca\") pod \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\" (UID: \"b88f0c6b-8e1a-46bf-a024-4e763458e7bc\") "
Oct 30 13:15:20.447640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.447626    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qg8l6\" (UniqueName: \"kubernetes.io/projected/0dfc3095-8c23-49a8-bc62-530830f57e13-kube-api-access-qg8l6\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:20.447723 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.447641    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0dfc3095-8c23-49a8-bc62-530830f57e13-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:20.447723 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.447697    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-maas-api-service-ca" pod "b88f0c6b-8e1a-46bf-a024-4e763458e7bc" (UID: "b88f0c6b-8e1a-46bf-a024-4e763458e7bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:20.449261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.449230    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-kube-api-access-pj7j4" pod "b88f0c6b-8e1a-46bf-a024-4e763458e7bc" (UID: "b88f0c6b-8e1a-46bf-a024-4e763458e7bc"). InnerVolumeSpecName "kube-api-access-pj7j4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:20.549415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.549338    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-maas-api-service-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:20.549415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.549364    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj7j4\" (UniqueName: \"kubernetes.io/projected/b88f0c6b-8e1a-46bf-a024-4e763458e7bc-kube-api-access-pj7j4\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:20.844511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:20.844421    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3425d5b-5541-40da-8bda-2c099588163e" path="/var/lib/kubelet/pods/e3425d5b-5541-40da-8bda-2c099588163e/volumes"
Oct 30 13:15:21.086773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.086721    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:21.105252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.105180    5781 generic.go:356] "Generic (PLEG): container finished" podID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerID="99cd70ce0eebae568a7a62c4c11f4f11b8d0732d8e25f4afa6c6a5967a43c75a" exitCode=60
Oct 30 13:15:21.105672 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.105253    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" event={"ID":"a2dd4b1d-794e-485c-8513-0db94b048a6c","Type":"ContainerDied","Data":"99cd70ce0eebae568a7a62c4c11f4f11b8d0732d8e25f4afa6c6a5967a43c75a"}
Oct 30 13:15:21.105672 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.105328    5781 scope.go:122] "RemoveContainer" containerID="a6b0a4fbb53d03cac447ce5831311f29b9a1e303ba0cccaafe216cff550fe01b"
Oct 30 13:15:21.105791 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.105703    5781 scope.go:122] "RemoveContainer" containerID="99cd70ce0eebae568a7a62c4c11f4f11b8d0732d8e25f4afa6c6a5967a43c75a"
Oct 30 13:15:21.106033 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:21.106014    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8_odh-ai-gateway-infra(a2dd4b1d-794e-485c-8513-0db94b048a6c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c"
Oct 30 13:15:21.109463 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.109445    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"
Oct 30 13:15:21.109572 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.109466    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx" event={"ID":"0dfc3095-8c23-49a8-bc62-530830f57e13","Type":"ContainerDied","Data":"45052d735aad71ccd44425cd28244628a6653e2fd57bd932ce1066403342e682"}
Oct 30 13:15:21.109572 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.109504    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45052d735aad71ccd44425cd28244628a6653e2fd57bd932ce1066403342e682"
Oct 30 13:15:21.110793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.110758    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp" event={"ID":"b88f0c6b-8e1a-46bf-a024-4e763458e7bc","Type":"ContainerDied","Data":"e0f0ea25bdcc41f0f0148e46ca1fd57fe3bf88d557e958bd80812778462cdc7c"}
Oct 30 13:15:21.110793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.110784    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"
Oct 30 13:15:21.119173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.119153    5781 scope.go:122] "RemoveContainer" containerID="9db932ae07ee92ef3a83325f35fc83af1c37e8fc39dc7cccc27e7c8af13995b0"
Oct 30 13:15:21.180479 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.180452    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"]
Oct 30 13:15:21.201661 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.201630    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"]
Oct 30 13:15:21.213977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.213946    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"]
Oct 30 13:15:21.215247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215223    5781 cpu_manager.go:414] "containerMap: removing container" podUID="e3425d5b-5541-40da-8bda-2c099588163e" containerName="cleanup"
Oct 30 13:15:21.215247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215248    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3425d5b-5541-40da-8bda-2c099588163e" containerName="cleanup"
Oct 30 13:15:21.215429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215260    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0dfc3095-8c23-49a8-bc62-530830f57e13" containerName="revoke-keys"
Oct 30 13:15:21.215429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215268    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dfc3095-8c23-49a8-bc62-530830f57e13" containerName="revoke-keys"
Oct 30 13:15:21.215429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215315    5781 cpu_manager.go:414] "containerMap: removing container" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerName="cleanup"
Oct 30 13:15:21.215429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215323    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerName="cleanup"
Oct 30 13:15:21.215429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215333    5781 cpu_manager.go:414] "containerMap: removing container" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerName="cleanup"
Oct 30 13:15:21.215429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215339    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerName="cleanup"
Oct 30 13:15:21.215429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215395    5781 cpu_manager.go:414] "containerMap: removing container" podUID="e3425d5b-5541-40da-8bda-2c099588163e" containerName="cleanup"
Oct 30 13:15:21.215429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215404    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3425d5b-5541-40da-8bda-2c099588163e" containerName="cleanup"
Oct 30 13:15:21.215680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215552    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3425d5b-5541-40da-8bda-2c099588163e" containerName="cleanup"
Oct 30 13:15:21.215680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215583    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dfc3095-8c23-49a8-bc62-530830f57e13" containerName="revoke-keys"
Oct 30 13:15:21.215680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215597    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3425d5b-5541-40da-8bda-2c099588163e" containerName="cleanup"
Oct 30 13:15:21.215680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215609    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerName="cleanup"
Oct 30 13:15:21.215680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215616    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerName="cleanup"
Oct 30 13:15:21.215680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.215627    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerName="cleanup"
Oct 30 13:15:21.221095 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.221074    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-031737-29889435-h6bdp"]
Oct 30 13:15:21.221201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.221162    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.242898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.242874    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-ab3176-openshift-default-dockercfg-ws7g2\""
Oct 30 13:15:21.253883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.253855    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"]
Oct 30 13:15:21.336026 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.336002    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:21.359896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.359820    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.360024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.359924    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-token\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.360024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.359983    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-credential-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.360024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.360018    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-podinfo\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.360162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.360084    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-data\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.360162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.360139    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-certs\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.360346 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.360327    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istiod-ca-cert\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.360416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.360374    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-envoy\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.360463 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.360425    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldq7n\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-kube-api-access-ldq7n\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.461226 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461193    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67a20054-fa55-4bd5-8871-42dfab7a611b-kube-api-access-qxklf\" (UniqueName: \"kubernetes.io/projected/67a20054-fa55-4bd5-8871-42dfab7a611b-kube-api-access-qxklf\") pod \"67a20054-fa55-4bd5-8871-42dfab7a611b\" (UID: \"67a20054-fa55-4bd5-8871-42dfab7a611b\") "
Oct 30 13:15:21.461412 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461359    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/67a20054-fa55-4bd5-8871-42dfab7a611b-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a20054-fa55-4bd5-8871-42dfab7a611b-maas-api-service-ca\") pod \"67a20054-fa55-4bd5-8871-42dfab7a611b\" (UID: \"67a20054-fa55-4bd5-8871-42dfab7a611b\") "
Oct 30 13:15:21.461515 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461491    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-envoy\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.461640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461518    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldq7n\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-kube-api-access-ldq7n\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.461640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461542    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.461640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461599    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-token\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.461804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461644    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-credential-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.461804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461682    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-podinfo\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.461804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461729    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-data\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.461804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461757    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-certs\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.461804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461765    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67a20054-fa55-4bd5-8871-42dfab7a611b-maas-api-service-ca" pod "67a20054-fa55-4bd5-8871-42dfab7a611b" (UID: "67a20054-fa55-4bd5-8871-42dfab7a611b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:21.462048 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461950    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istiod-ca-cert\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.462048 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.461954    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.462149 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.462055    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-certs\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.462149 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.462139    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-credential-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.462231 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.462156    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67a20054-fa55-4bd5-8871-42dfab7a611b-maas-api-service-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:21.462452 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.462431    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-data\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.462777 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.462748    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istiod-ca-cert\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.463608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.463553    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67a20054-fa55-4bd5-8871-42dfab7a611b-kube-api-access-qxklf" pod "67a20054-fa55-4bd5-8871-42dfab7a611b" (UID: "67a20054-fa55-4bd5-8871-42dfab7a611b"). InnerVolumeSpecName "kube-api-access-qxklf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:21.463916 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.463900    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-envoy\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.464240 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.464224    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-podinfo\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.503046 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.503014    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldq7n\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-kube-api-access-ldq7n\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.503509 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.503484    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-token\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.534008 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.533972    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:21.570764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.564283    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxklf\" (UniqueName: \"kubernetes.io/projected/67a20054-fa55-4bd5-8871-42dfab7a611b-kube-api-access-qxklf\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:21.946311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.940005    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"]
Oct 30 13:15:21.963817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.962187    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:15:21.963817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.962299    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:15:21.963817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:21.962696    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:15:22.135308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.134194    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" event={"ID":"9e8a198f-e34c-4216-bf22-ed9cf03d91f4","Type":"ContainerStarted","Data":"e91e47d20a28194eb635b81c5c2de9664cd6f63c5cbfb26c6f87a679c94d4b3d"}
Oct 30 13:15:22.135308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.134265    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" event={"ID":"9e8a198f-e34c-4216-bf22-ed9cf03d91f4","Type":"ContainerStarted","Data":"76ffe8613da36cb8cd4cc9d3fa8654606ae33574ac7543e93119645e32a5ae41"}
Oct 30 13:15:22.164183 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.164089    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q" event={"ID":"67a20054-fa55-4bd5-8871-42dfab7a611b","Type":"ContainerDied","Data":"2fa991a81912aa7ea7153513d1bffbb68ddac394327bd7a49bef202aa5890330"}
Oct 30 13:15:22.164475 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.164441    5781 scope.go:122] "RemoveContainer" containerID="209f42eef21812375b98a09a166775647769eab347f8de4f0f5f41fe63ef3e31"
Oct 30 13:15:22.164760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.164729    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"
Oct 30 13:15:22.210456 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.210430    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"]
Oct 30 13:15:22.298434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.298365    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" podStartSLOduration=1.298341319 podStartE2EDuration="1.298341319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:22.255869059 +0000 UTC m=+1059.553086601" watchObservedRunningTime="2026-10-30 13:15:22.298341319 +0000 UTC m=+1059.595558864"
Oct 30 13:15:22.374440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.374406    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"]
Oct 30 13:15:22.446396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.431901    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-f76c09-29889435-hn42q"]
Oct 30 13:15:22.574313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.543735    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:22.819080 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.819055    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:22.847849 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.847815    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" path="/var/lib/kubelet/pods/67a20054-fa55-4bd5-8871-42dfab7a611b/volumes"
Oct 30 13:15:22.848375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.848360    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" path="/var/lib/kubelet/pods/b88f0c6b-8e1a-46bf-a024-4e763458e7bc/volumes"
Oct 30 13:15:22.887671 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.887282    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2dd4b1d-794e-485c-8513-0db94b048a6c-kube-api-access-r4qpg\" (UniqueName: \"kubernetes.io/projected/a2dd4b1d-794e-485c-8513-0db94b048a6c-kube-api-access-r4qpg\") pod \"a2dd4b1d-794e-485c-8513-0db94b048a6c\" (UID: \"a2dd4b1d-794e-485c-8513-0db94b048a6c\") "
Oct 30 13:15:22.887671 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.887409    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2dd4b1d-794e-485c-8513-0db94b048a6c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2dd4b1d-794e-485c-8513-0db94b048a6c-maas-api-service-ca\") pod \"a2dd4b1d-794e-485c-8513-0db94b048a6c\" (UID: \"a2dd4b1d-794e-485c-8513-0db94b048a6c\") "
Oct 30 13:15:22.906382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.904062    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2dd4b1d-794e-485c-8513-0db94b048a6c-maas-api-service-ca" pod "a2dd4b1d-794e-485c-8513-0db94b048a6c" (UID: "a2dd4b1d-794e-485c-8513-0db94b048a6c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:22.937308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.914834    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2dd4b1d-794e-485c-8513-0db94b048a6c-kube-api-access-r4qpg" pod "a2dd4b1d-794e-485c-8513-0db94b048a6c" (UID: "a2dd4b1d-794e-485c-8513-0db94b048a6c"). InnerVolumeSpecName "kube-api-access-r4qpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:22.989345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.989308    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2dd4b1d-794e-485c-8513-0db94b048a6c-maas-api-service-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:22.989345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:22.989333    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4qpg\" (UniqueName: \"kubernetes.io/projected/a2dd4b1d-794e-485c-8513-0db94b048a6c-kube-api-access-r4qpg\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:23.172101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:23.172067    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"
Oct 30 13:15:23.172101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:23.172077    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8" event={"ID":"a2dd4b1d-794e-485c-8513-0db94b048a6c","Type":"ContainerDied","Data":"e382b1b9354b75c4195019d022a64eec9808058061e38e31acdbeb55dfe337ea"}
Oct 30 13:15:23.172633 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:23.172143    5781 scope.go:122] "RemoveContainer" containerID="99cd70ce0eebae568a7a62c4c11f4f11b8d0732d8e25f4afa6c6a5967a43c75a"
Oct 30 13:15:23.239473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:23.239432    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"]
Oct 30 13:15:23.249693 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:23.249661    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-88f6a4-29889435-gbks8"]
Oct 30 13:15:23.307529 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:23.307495    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:23.538915 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:23.538822    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 13:15:23.538915 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:23.538888    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 13:15:24.537698 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:24.537658    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 13:15:24.538168 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:24.537718    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 13:15:24.844455 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:24.844379    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" path="/var/lib/kubelet/pods/a2dd4b1d-794e-485c-8513-0db94b048a6c/volumes"
Oct 30 13:15:25.033888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:25.033832    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:25.534708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:25.534671    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 13:15:25.534981 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:25.534953    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 13:15:26.537713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.537674    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 13:15:26.538230 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.537724    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 13:15:26.681550 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.681506    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"]
Oct 30 13:15:26.682927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.682899    5781 cpu_manager.go:414] "containerMap: removing container" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerName="cleanup"
Oct 30 13:15:26.682927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.682926    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.682949    5781 cpu_manager.go:414] "containerMap: removing container" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.682957    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.682986    5781 cpu_manager.go:414] "containerMap: removing container" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.682993    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683021    5781 cpu_manager.go:414] "containerMap: removing container" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683028    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683055    5781 cpu_manager.go:414] "containerMap: removing container" podUID="e3425d5b-5541-40da-8bda-2c099588163e" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683062    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3425d5b-5541-40da-8bda-2c099588163e" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683100    5781 cpu_manager.go:414] "containerMap: removing container" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerName="cleanup"
Oct 30 13:15:26.683111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683107    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b88f0c6b-8e1a-46bf-a024-4e763458e7bc" containerName="cleanup"
Oct 30 13:15:26.683563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683395    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerName="cleanup"
Oct 30 13:15:26.683563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683417    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerName="cleanup"
Oct 30 13:15:26.683563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683429    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerName="cleanup"
Oct 30 13:15:26.683563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683438    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerName="cleanup"
Oct 30 13:15:26.683563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683459    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3425d5b-5541-40da-8bda-2c099588163e" containerName="cleanup"
Oct 30 13:15:26.683563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683467    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerName="cleanup"
Oct 30 13:15:26.683563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.683474    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerName="cleanup"
Oct 30 13:15:26.693587 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.693563    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:26.702922 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.702890    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"]
Oct 30 13:15:26.711398 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.711381    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:26.711512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.711385    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:26.741615 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.741576    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:26.857136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.857061    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4b4a013c-9wh6m\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:26.857265 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.857230    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4b4a013c-9wh6m\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:26.857340 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.857262    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcmp8\" (UniqueName: \"kubernetes.io/projected/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-kube-api-access-fcmp8\") pod \"maas-api-revoke-sub-4b4a013c-9wh6m\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:26.958994 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.958963    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4b4a013c-9wh6m\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:26.959157 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.959004    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcmp8\" (UniqueName: \"kubernetes.io/projected/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-kube-api-access-fcmp8\") pod \"maas-api-revoke-sub-4b4a013c-9wh6m\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:26.959157 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.959106    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4b4a013c-9wh6m\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:26.959734 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.959709    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4b4a013c-9wh6m\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:26.961437 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.961415    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4b4a013c-9wh6m\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:26.982504 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:26.982471    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcmp8\" (UniqueName: \"kubernetes.io/projected/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-kube-api-access-fcmp8\") pod \"maas-api-revoke-sub-4b4a013c-9wh6m\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:27.009121 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:27.009102    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:27.349032 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:27.348964    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2ff6ac5_e9c1_47bf_82b8_6c818dd0294f.slice/crio-c1a2c2f899d9d4bc8dfe4ce6f252b40b673c0db4027d919c8db49608a5ddb0b4 WatchSource:0}: Error finding container c1a2c2f899d9d4bc8dfe4ce6f252b40b673c0db4027d919c8db49608a5ddb0b4: Status 404 returned error can't find the container with id c1a2c2f899d9d4bc8dfe4ce6f252b40b673c0db4027d919c8db49608a5ddb0b4
Oct 30 13:15:27.349159 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:27.349096    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"]
Oct 30 13:15:27.538826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:27.538789    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 13:15:27.539267 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:27.538851    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 13:15:28.205174 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:28.205102    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m" event={"ID":"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f","Type":"ContainerStarted","Data":"e5941838859a6fc7cdf7283c5fe5d3e37ac740be7a4daae86ae71efe9c161476"}
Oct 30 13:15:28.205174 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:28.205159    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m" event={"ID":"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f","Type":"ContainerStarted","Data":"c1a2c2f899d9d4bc8dfe4ce6f252b40b673c0db4027d919c8db49608a5ddb0b4"}
Oct 30 13:15:28.251975 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:28.251923    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m" podStartSLOduration=2.251907445 podStartE2EDuration="2.251907445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:28.251550522 +0000 UTC m=+1065.548768064" watchObservedRunningTime="2026-10-30 13:15:28.251907445 +0000 UTC m=+1065.549124987"
Oct 30 13:15:28.370730 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:28.370696    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:28.534649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:28.534565    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 13:15:28.534649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:28.534622    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 13:15:29.212937 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:29.212902    5781 generic.go:356] "Generic (PLEG): container finished" podID="d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f" containerID="e5941838859a6fc7cdf7283c5fe5d3e37ac740be7a4daae86ae71efe9c161476" exitCode=0
Oct 30 13:15:29.213301 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:29.212958    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m" event={"ID":"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f","Type":"ContainerDied","Data":"e5941838859a6fc7cdf7283c5fe5d3e37ac740be7a4daae86ae71efe9c161476"}
Oct 30 13:15:29.538519 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:29.538449    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:29.538848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:29.538830    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:29.539458 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:29.539443    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:15:30.090659 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.090604    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:30.378542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.378498    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:30.501151 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.501124    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-kube-api-access-fcmp8\" (UniqueName: \"kubernetes.io/projected/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-kube-api-access-fcmp8\") pod \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") "
Oct 30 13:15:30.501282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.501173    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-xks-ca\") pod \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") "
Oct 30 13:15:30.501282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.501194    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-openshift-ca\") pod \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\" (UID: \"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f\") "
Oct 30 13:15:30.501641 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.501606    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-openshift-ca" pod "d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f" (UID: "d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:30.503201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.503180    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-kube-api-access-fcmp8" pod "d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f" (UID: "d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f"). InnerVolumeSpecName "kube-api-access-fcmp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:30.503313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.503268    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-xks-ca" pod "d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f" (UID: "d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:30.603193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.603136    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcmp8\" (UniqueName: \"kubernetes.io/projected/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-kube-api-access-fcmp8\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:30.603193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.603159    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:30.603193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.603169    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:30.936252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:30.936226    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:31.229458 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:31.229379    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"
Oct 30 13:15:31.229458 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:31.229419    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m" event={"ID":"d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f","Type":"ContainerDied","Data":"c1a2c2f899d9d4bc8dfe4ce6f252b40b673c0db4027d919c8db49608a5ddb0b4"}
Oct 30 13:15:31.229613 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:31.229472    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1a2c2f899d9d4bc8dfe4ce6f252b40b673c0db4027d919c8db49608a5ddb0b4"
Oct 30 13:15:31.751506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:31.751469    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:31.996208 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:15:31.996172    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:15:33.327127 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.327083    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:33.515085 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.515051    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"]
Oct 30 13:15:33.515763 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.515745    5781 cpu_manager.go:414] "containerMap: removing container" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerName="cleanup"
Oct 30 13:15:33.515826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.515764    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67a20054-fa55-4bd5-8871-42dfab7a611b" containerName="cleanup"
Oct 30 13:15:33.515826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.515778    5781 cpu_manager.go:414] "containerMap: removing container" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerName="cleanup"
Oct 30 13:15:33.515826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.515784    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2dd4b1d-794e-485c-8513-0db94b048a6c" containerName="cleanup"
Oct 30 13:15:33.515826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.515802    5781 cpu_manager.go:414] "containerMap: removing container" podUID="d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f" containerName="revoke-keys"
Oct 30 13:15:33.515826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.515807    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f" containerName="revoke-keys"
Oct 30 13:15:33.515977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.515897    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f" containerName="revoke-keys"
Oct 30 13:15:33.521049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.521026    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.535025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.535003    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:33.537685 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.537658    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16400144-2b71-4456-827d-046c6575a757-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-92666b8c-9bgc6\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.537822 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.537720    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7sdq\" (UniqueName: \"kubernetes.io/projected/16400144-2b71-4456-827d-046c6575a757-kube-api-access-r7sdq\") pod \"maas-api-revoke-sub-92666b8c-9bgc6\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.537822 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.537785    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16400144-2b71-4456-827d-046c6575a757-maas-api-xks-ca\") pod \"maas-api-revoke-sub-92666b8c-9bgc6\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.546590 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.546543    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:33.551336 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.550599    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"]
Oct 30 13:15:33.639058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.638985    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16400144-2b71-4456-827d-046c6575a757-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-92666b8c-9bgc6\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.639190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.639068    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7sdq\" (UniqueName: \"kubernetes.io/projected/16400144-2b71-4456-827d-046c6575a757-kube-api-access-r7sdq\") pod \"maas-api-revoke-sub-92666b8c-9bgc6\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.639190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.639091    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16400144-2b71-4456-827d-046c6575a757-maas-api-xks-ca\") pod \"maas-api-revoke-sub-92666b8c-9bgc6\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.639777 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.639752    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16400144-2b71-4456-827d-046c6575a757-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-92666b8c-9bgc6\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.641487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.641464    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16400144-2b71-4456-827d-046c6575a757-maas-api-xks-ca\") pod \"maas-api-revoke-sub-92666b8c-9bgc6\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.661111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.661086    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7sdq\" (UniqueName: \"kubernetes.io/projected/16400144-2b71-4456-827d-046c6575a757-kube-api-access-r7sdq\") pod \"maas-api-revoke-sub-92666b8c-9bgc6\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:33.836371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:33.836337    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:34.140551 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:34.140474    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"]
Oct 30 13:15:34.144493 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:34.144463    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16400144_2b71_4456_827d_046c6575a757.slice/crio-a8266a6dfbd67d1c4ac31a5bd04fd4ed6939924879d1b3bf754c364c88fe652e WatchSource:0}: Error finding container a8266a6dfbd67d1c4ac31a5bd04fd4ed6939924879d1b3bf754c364c88fe652e: Status 404 returned error can't find the container with id a8266a6dfbd67d1c4ac31a5bd04fd4ed6939924879d1b3bf754c364c88fe652e
Oct 30 13:15:34.246685 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:34.246656    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6" event={"ID":"16400144-2b71-4456-827d-046c6575a757","Type":"ContainerStarted","Data":"a8266a6dfbd67d1c4ac31a5bd04fd4ed6939924879d1b3bf754c364c88fe652e"}
Oct 30 13:15:34.781903 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:34.781826    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:35.253583 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.253539    5781 generic.go:356] "Generic (PLEG): container finished" podID="16400144-2b71-4456-827d-046c6575a757" containerID="83902f869ffa19546c1426c665e05004721844848f5c73eae1be76903efe5f5d" exitCode=0
Oct 30 13:15:35.253757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.253624    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6" event={"ID":"16400144-2b71-4456-827d-046c6575a757","Type":"ContainerDied","Data":"83902f869ffa19546c1426c665e05004721844848f5c73eae1be76903efe5f5d"}
Oct 30 13:15:35.801331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.801279    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"]
Oct 30 13:15:35.805830 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.805809    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.842165 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.842137    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"]
Oct 30 13:15:35.864994 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.864956    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.865115 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.865009    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.865153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.865115    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d5455b9-c745-4c33-9599-63fbb962dd0d-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.865203 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.865183    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-548rw\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-kube-api-access-548rw\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.865243 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.865234    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.865414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.865388    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.865457 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.865432    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.865540 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.865519    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.865613 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.865597    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967248 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967223    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967263    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967313    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d5455b9-c745-4c33-9599-63fbb962dd0d-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967340    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-548rw\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-kube-api-access-548rw\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967374    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967431    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967460    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967509    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967564    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967618    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967665    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967768    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.967950 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967886    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.968015 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.967993    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d5455b9-c745-4c33-9599-63fbb962dd0d-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.969748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.969721    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.969931 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.969916    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.990444 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.990420    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:35.990949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:35.990930    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-548rw\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-kube-api-access-548rw\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:36.065140 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.065065    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"]
Oct 30 13:15:36.070496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.070466    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.085565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.085548    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-f8a55e-openshift-default-dockercfg-69hv5\""
Oct 30 13:15:36.108351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.108324    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"]
Oct 30 13:15:36.120512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.120475    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:36.170022 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.169976    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.170022 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.170022    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.170275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.170115    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zz2hf\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-kube-api-access-zz2hf\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.170275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.170200    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.170275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.170217    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.170275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.170241    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.170506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.170369    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.170506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.170426    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.170571 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.170529    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271276    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271449 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271336    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271449 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271365    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271449 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271435    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271603 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271528    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271660 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271624    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271660 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271651    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271675    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271690    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271756    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zz2hf\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-kube-api-access-zz2hf\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.271975 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.271894    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.272048 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.272024    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.272390 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.272368    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.272613 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.272591    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.273867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.273847    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.274448 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.274431    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.296728 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.296706    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.296829 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.296763    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zz2hf\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-kube-api-access-zz2hf\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.386059 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.386031    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:36.389246 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.389207    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:36.391239 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.389510    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:36.453323 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.453279    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"]
Oct 30 13:15:36.457788 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:36.457388    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5455b9_c745_4c33_9599_63fbb962dd0d.slice/crio-f8a24fa1089c3346e95479e97ed72245ee634e2ce9081545cc502fd098aea4ca WatchSource:0}: Error finding container f8a24fa1089c3346e95479e97ed72245ee634e2ce9081545cc502fd098aea4ca: Status 404 returned error can't find the container with id f8a24fa1089c3346e95479e97ed72245ee634e2ce9081545cc502fd098aea4ca
Oct 30 13:15:36.460225 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.460178    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:15:36.460389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.460281    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:15:36.460605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.460584    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:15:36.473895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.473868    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/16400144-2b71-4456-827d-046c6575a757-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16400144-2b71-4456-827d-046c6575a757-maas-api-xks-ca\") pod \"16400144-2b71-4456-827d-046c6575a757\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") "
Oct 30 13:15:36.474074 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.474048    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/16400144-2b71-4456-827d-046c6575a757-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16400144-2b71-4456-827d-046c6575a757-maas-api-openshift-ca\") pod \"16400144-2b71-4456-827d-046c6575a757\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") "
Oct 30 13:15:36.474198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.474174    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16400144-2b71-4456-827d-046c6575a757-kube-api-access-r7sdq\" (UniqueName: \"kubernetes.io/projected/16400144-2b71-4456-827d-046c6575a757-kube-api-access-r7sdq\") pod \"16400144-2b71-4456-827d-046c6575a757\" (UID: \"16400144-2b71-4456-827d-046c6575a757\") "
Oct 30 13:15:36.474854 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.474715    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16400144-2b71-4456-827d-046c6575a757-maas-api-openshift-ca" pod "16400144-2b71-4456-827d-046c6575a757" (UID: "16400144-2b71-4456-827d-046c6575a757"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:36.477693 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.477671    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16400144-2b71-4456-827d-046c6575a757-maas-api-xks-ca" pod "16400144-2b71-4456-827d-046c6575a757" (UID: "16400144-2b71-4456-827d-046c6575a757"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:36.477774 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.477694    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16400144-2b71-4456-827d-046c6575a757-kube-api-access-r7sdq" pod "16400144-2b71-4456-827d-046c6575a757" (UID: "16400144-2b71-4456-827d-046c6575a757"). InnerVolumeSpecName "kube-api-access-r7sdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:36.576493 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.576458    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/16400144-2b71-4456-827d-046c6575a757-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:36.576493 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.576491    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7sdq\" (UniqueName: \"kubernetes.io/projected/16400144-2b71-4456-827d-046c6575a757-kube-api-access-r7sdq\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:36.576755 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.576507    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/16400144-2b71-4456-827d-046c6575a757-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:36.717177 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.717144    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"]
Oct 30 13:15:36.720796 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:36.720770    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c0e318_a41a_4edb_b0e6_9652cf048d9a.slice/crio-4038fe1d82d02a5d45e4076c0843f8199bad4bad3b1807c5fbbda2fb067b14fd WatchSource:0}: Error finding container 4038fe1d82d02a5d45e4076c0843f8199bad4bad3b1807c5fbbda2fb067b14fd: Status 404 returned error can't find the container with id 4038fe1d82d02a5d45e4076c0843f8199bad4bad3b1807c5fbbda2fb067b14fd
Oct 30 13:15:36.723257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.723226    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:15:36.723370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.723308    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:15:36.723370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:36.723344    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:15:37.275935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.275836    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" event={"ID":"2d5455b9-c745-4c33-9599-63fbb962dd0d","Type":"ContainerStarted","Data":"8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb"}
Oct 30 13:15:37.276563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.276542    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" event={"ID":"2d5455b9-c745-4c33-9599-63fbb962dd0d","Type":"ContainerStarted","Data":"f8a24fa1089c3346e95479e97ed72245ee634e2ce9081545cc502fd098aea4ca"}
Oct 30 13:15:37.277519 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.277499    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"
Oct 30 13:15:37.277519 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.277503    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6" event={"ID":"16400144-2b71-4456-827d-046c6575a757","Type":"ContainerDied","Data":"a8266a6dfbd67d1c4ac31a5bd04fd4ed6939924879d1b3bf754c364c88fe652e"}
Oct 30 13:15:37.277701 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.277549    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8266a6dfbd67d1c4ac31a5bd04fd4ed6939924879d1b3bf754c364c88fe652e"
Oct 30 13:15:37.288173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.288138    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" event={"ID":"76c0e318-a41a-4edb-b0e6-9652cf048d9a","Type":"ContainerStarted","Data":"f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d"}
Oct 30 13:15:37.288374 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.288354    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" event={"ID":"76c0e318-a41a-4edb-b0e6-9652cf048d9a","Type":"ContainerStarted","Data":"4038fe1d82d02a5d45e4076c0843f8199bad4bad3b1807c5fbbda2fb067b14fd"}
Oct 30 13:15:37.358650 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.358584    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" podStartSLOduration=2.358561712 podStartE2EDuration="2.358561712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:37.328653723 +0000 UTC m=+1074.625871276" watchObservedRunningTime="2026-10-30 13:15:37.358561712 +0000 UTC m=+1074.655779256"
Oct 30 13:15:37.376857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.376805    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podStartSLOduration=1.376788648 podStartE2EDuration="1.376788648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:37.376233812 +0000 UTC m=+1074.673451379" watchObservedRunningTime="2026-10-30 13:15:37.376788648 +0000 UTC m=+1074.674006189"
Oct 30 13:15:37.387138 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.387104    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:37.388979 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.388947    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 13:15:37.389121 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:37.389006    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 13:15:38.136312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:38.135348    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:38.136630 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:38.136606    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 13:15:38.136837 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:38.136810    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 13:15:38.387363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:38.386654    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 13:15:38.387363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:38.386724    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 13:15:38.623312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:38.613382    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:38.748910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:38.748865    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:38.842093 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:38.842052    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:38.975329 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:38.970111    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:39.008658 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.008388    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:39.131747 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.131706    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 13:15:39.132034 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.132003    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 13:15:39.309364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.308361    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:39.412447 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.412411    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 13:15:39.413053 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.413005    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 13:15:39.619568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.619438    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:39.620755 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.620736    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:15:39.637546 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.637508    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:39.700752 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:39.700710    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:40.013318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.012530    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:40.013318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.013231    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:15:40.122128 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.121715    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 13:15:40.122128 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.121770    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 13:15:40.345962 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.345862    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:40.346976 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.346957    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:15:40.410162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.409345    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 13:15:40.410162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.409408    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 13:15:40.647364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.620973    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:40.715743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.711247    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 13:15:40.717447 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.717422    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:15:40.863310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:40.859387    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 13:15:41.028620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.028579    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:41.135799 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.135755    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 13:15:41.151310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.147473    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 13:15:41.405378 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.404540    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": context deadline exceeded"
Oct 30 13:15:41.458311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.456616    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 13:15:41.458311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.456679    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 13:15:41.624452 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.621402    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:41.690414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.666802    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:41.717310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.705527    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:41.718678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.717645    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:41.868462 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.868410    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:41.869507 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.869485    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:15:41.902231 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.902198    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:15:41.994049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:41.993956    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:15:42.072353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:42.058370    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:15:42.078964 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:42.078934    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:15:42.095845 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:42.095815    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:15:42.098723 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:42.098701    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:15:42.173665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:42.173594    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 13:15:42.173665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:42.173650    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 13:15:42.387533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:42.387430    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 13:15:42.387533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:42.387486    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 13:15:43.107889 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.107862    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"]
Oct 30 13:15:43.108604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.108584    5781 cpu_manager.go:414] "containerMap: removing container" podUID="16400144-2b71-4456-827d-046c6575a757" containerName="revoke-keys"
Oct 30 13:15:43.108704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.108604    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16400144-2b71-4456-827d-046c6575a757" containerName="revoke-keys"
Oct 30 13:15:43.108812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.108798    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="16400144-2b71-4456-827d-046c6575a757" containerName="revoke-keys"
Oct 30 13:15:43.138609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.138572    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.139678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.139650    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 13:15:43.139899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.139873    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 13:15:43.173473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.165942    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:43.173473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.171951    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"]
Oct 30 13:15:43.177193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.177165    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:43.289668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.289591    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qjg9\" (UniqueName: \"kubernetes.io/projected/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-kube-api-access-2qjg9\") pod \"maas-api-revoke-sub-68091082-r5245\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.289804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.289718    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-68091082-r5245\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.289843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.289807    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-xks-ca\") pod \"maas-api-revoke-sub-68091082-r5245\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.387023 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.386988    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 13:15:43.387204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.387048    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 13:15:43.390742 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.390712    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qjg9\" (UniqueName: \"kubernetes.io/projected/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-kube-api-access-2qjg9\") pod \"maas-api-revoke-sub-68091082-r5245\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.390871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.390792    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-68091082-r5245\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.390871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.390863    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-xks-ca\") pod \"maas-api-revoke-sub-68091082-r5245\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.391485 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.391455    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-68091082-r5245\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.393161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.393136    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-xks-ca\") pod \"maas-api-revoke-sub-68091082-r5245\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.447532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.445764    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qjg9\" (UniqueName: \"kubernetes.io/projected/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-kube-api-access-2qjg9\") pod \"maas-api-revoke-sub-68091082-r5245\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.482392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.482365    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:43.844141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.843850    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"]
Oct 30 13:15:43.847277 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:43.847250    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c2b5c11_ccc4_4d04_b999_4c7ee1501e24.slice/crio-20c08ab159ed8c29d736ffbae44c23c02929fc686a0ef8ee9d9182d97e57d6ca WatchSource:0}: Error finding container 20c08ab159ed8c29d736ffbae44c23c02929fc686a0ef8ee9d9182d97e57d6ca: Status 404 returned error can't find the container with id 20c08ab159ed8c29d736ffbae44c23c02929fc686a0ef8ee9d9182d97e57d6ca
Oct 30 13:15:43.969106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:43.969064    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:44.387121 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:44.387078    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 13:15:44.387657 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:44.387147    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 13:15:44.600828 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:44.600719    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245" event={"ID":"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24","Type":"ContainerStarted","Data":"d838f6e9664f6c0b59a8e8ecac64fa0a6096f2fee62783096aa439bc91c6526c"}
Oct 30 13:15:44.601012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:44.600827    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245" event={"ID":"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24","Type":"ContainerStarted","Data":"20c08ab159ed8c29d736ffbae44c23c02929fc686a0ef8ee9d9182d97e57d6ca"}
Oct 30 13:15:44.780097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:44.780042    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245" podStartSLOduration=1.78002734 podStartE2EDuration="1.78002734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:44.776978141 +0000 UTC m=+1082.074195694" watchObservedRunningTime="2026-10-30 13:15:44.78002734 +0000 UTC m=+1082.077244881"
Oct 30 13:15:44.960304 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:44.960257    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:44.960830 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:44.960788    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:44.961571 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:44.961540    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:15:45.391563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:45.391537    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:45.391928 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:45.391894    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:45.392715 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:45.392665    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:15:45.607701 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:45.607673    5781 generic.go:356] "Generic (PLEG): container finished" podID="3c2b5c11-ccc4-4d04-b999-4c7ee1501e24" containerID="d838f6e9664f6c0b59a8e8ecac64fa0a6096f2fee62783096aa439bc91c6526c" exitCode=0
Oct 30 13:15:45.607898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:45.607758    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245" event={"ID":"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24","Type":"ContainerDied","Data":"d838f6e9664f6c0b59a8e8ecac64fa0a6096f2fee62783096aa439bc91c6526c"}
Oct 30 13:15:46.428455 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.428423    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:46.755770 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.755748    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:46.845790 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.845763    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-openshift-ca\") pod \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") "
Oct 30 13:15:46.845926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.845801    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-xks-ca\") pod \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") "
Oct 30 13:15:46.845989 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.845974    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-kube-api-access-2qjg9\" (UniqueName: \"kubernetes.io/projected/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-kube-api-access-2qjg9\") pod \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\" (UID: \"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24\") "
Oct 30 13:15:46.846243 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.846211    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-openshift-ca" pod "3c2b5c11-ccc4-4d04-b999-4c7ee1501e24" (UID: "3c2b5c11-ccc4-4d04-b999-4c7ee1501e24"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:46.846529 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.846452    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:46.848545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.848523    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-xks-ca" pod "3c2b5c11-ccc4-4d04-b999-4c7ee1501e24" (UID: "3c2b5c11-ccc4-4d04-b999-4c7ee1501e24"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:46.848642 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.848575    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-kube-api-access-2qjg9" pod "3c2b5c11-ccc4-4d04-b999-4c7ee1501e24" (UID: "3c2b5c11-ccc4-4d04-b999-4c7ee1501e24"). InnerVolumeSpecName "kube-api-access-2qjg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:46.948036 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.948008    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qjg9\" (UniqueName: \"kubernetes.io/projected/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-kube-api-access-2qjg9\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:46.948036 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:46.948032    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:47.621369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:47.621321    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245" event={"ID":"3c2b5c11-ccc4-4d04-b999-4c7ee1501e24","Type":"ContainerDied","Data":"20c08ab159ed8c29d736ffbae44c23c02929fc686a0ef8ee9d9182d97e57d6ca"}
Oct 30 13:15:47.621369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:47.621340    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"
Oct 30 13:15:47.621924 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:47.621402    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="20c08ab159ed8c29d736ffbae44c23c02929fc686a0ef8ee9d9182d97e57d6ca"
Oct 30 13:15:48.124515 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:48.124472    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:48.985314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:48.985272    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"]
Oct 30 13:15:48.985857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:48.985844    5781 cpu_manager.go:414] "containerMap: removing container" podUID="3c2b5c11-ccc4-4d04-b999-4c7ee1501e24" containerName="revoke-keys"
Oct 30 13:15:48.985910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:48.985858    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c2b5c11-ccc4-4d04-b999-4c7ee1501e24" containerName="revoke-keys"
Oct 30 13:15:48.985972 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:48.985963    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c2b5c11-ccc4-4d04-b999-4c7ee1501e24" containerName="revoke-keys"
Oct 30 13:15:48.989795 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:48.989777    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.019760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.019733    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:49.023068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.023052    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:49.037406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.037383    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"]
Oct 30 13:15:49.072689 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.072655    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ab936e4a-vcs6x\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.072888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.072696    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ab936e4a-vcs6x\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.072888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.072721    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/0297f575-381a-4fed-ad28-6d257c127d64-kube-api-access-bdpgc\") pod \"maas-api-revoke-sub-ab936e4a-vcs6x\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.173859 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.173830    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ab936e4a-vcs6x\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.173859 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.173862    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ab936e4a-vcs6x\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.174105 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.173882    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/0297f575-381a-4fed-ad28-6d257c127d64-kube-api-access-bdpgc\") pod \"maas-api-revoke-sub-ab936e4a-vcs6x\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.174576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.174550    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-ab936e4a-vcs6x\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.176232 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.176212    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-xks-ca\") pod \"maas-api-revoke-sub-ab936e4a-vcs6x\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.197786 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.197752    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/0297f575-381a-4fed-ad28-6d257c127d64-kube-api-access-bdpgc\") pod \"maas-api-revoke-sub-ab936e4a-vcs6x\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.304000 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.303913    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:49.607595 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.607517    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"]
Oct 30 13:15:49.610884 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:49.610861    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0297f575_381a_4fed_ad28_6d257c127d64.slice/crio-054a9d1ab124f19d394d4241a7ddafc2ab7f37a3add534fcdd65ca74f8665c12 WatchSource:0}: Error finding container 054a9d1ab124f19d394d4241a7ddafc2ab7f37a3add534fcdd65ca74f8665c12: Status 404 returned error can't find the container with id 054a9d1ab124f19d394d4241a7ddafc2ab7f37a3add534fcdd65ca74f8665c12
Oct 30 13:15:49.635081 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.635054    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x" event={"ID":"0297f575-381a-4fed-ad28-6d257c127d64","Type":"ContainerStarted","Data":"054a9d1ab124f19d394d4241a7ddafc2ab7f37a3add534fcdd65ca74f8665c12"}
Oct 30 13:15:49.823644 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:49.823609    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:50.642860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:50.642825    5781 generic.go:356] "Generic (PLEG): container finished" podID="0297f575-381a-4fed-ad28-6d257c127d64" containerID="93736a243db39b778ce165e9ddd52f5e1ac03b301ddd14d0ea1cceaaf9d05eb3" exitCode=0
Oct 30 13:15:50.643315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:50.642909    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x" event={"ID":"0297f575-381a-4fed-ad28-6d257c127d64","Type":"ContainerDied","Data":"93736a243db39b778ce165e9ddd52f5e1ac03b301ddd14d0ea1cceaaf9d05eb3"}
Oct 30 13:15:51.784007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:51.783979    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:51.815987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:51.815961    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:51.909103 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:51.909031    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-openshift-ca\") pod \"0297f575-381a-4fed-ad28-6d257c127d64\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") "
Oct 30 13:15:51.909240 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:51.909229    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0297f575-381a-4fed-ad28-6d257c127d64-kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/0297f575-381a-4fed-ad28-6d257c127d64-kube-api-access-bdpgc\") pod \"0297f575-381a-4fed-ad28-6d257c127d64\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") "
Oct 30 13:15:51.909417 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:51.909395    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-openshift-ca" pod "0297f575-381a-4fed-ad28-6d257c127d64" (UID: "0297f575-381a-4fed-ad28-6d257c127d64"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:51.909488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:51.909471    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-xks-ca\") pod \"0297f575-381a-4fed-ad28-6d257c127d64\" (UID: \"0297f575-381a-4fed-ad28-6d257c127d64\") "
Oct 30 13:15:51.909882 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:51.909861    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:51.911326 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:51.911298    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-xks-ca" pod "0297f575-381a-4fed-ad28-6d257c127d64" (UID: "0297f575-381a-4fed-ad28-6d257c127d64"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:51.911409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:51.911326    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0297f575-381a-4fed-ad28-6d257c127d64-kube-api-access-bdpgc" pod "0297f575-381a-4fed-ad28-6d257c127d64" (UID: "0297f575-381a-4fed-ad28-6d257c127d64"). InnerVolumeSpecName "kube-api-access-bdpgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:52.011498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:52.011469    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0297f575-381a-4fed-ad28-6d257c127d64-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:52.011498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:52.011494    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdpgc\" (UniqueName: \"kubernetes.io/projected/0297f575-381a-4fed-ad28-6d257c127d64-kube-api-access-bdpgc\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:52.659522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:52.659493    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"
Oct 30 13:15:52.659522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:52.659505    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x" event={"ID":"0297f575-381a-4fed-ad28-6d257c127d64","Type":"ContainerDied","Data":"054a9d1ab124f19d394d4241a7ddafc2ab7f37a3add534fcdd65ca74f8665c12"}
Oct 30 13:15:52.659732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:52.659603    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="054a9d1ab124f19d394d4241a7ddafc2ab7f37a3add534fcdd65ca74f8665c12"
Oct 30 13:15:53.381882 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.381417    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:53.635127 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.635053    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"]
Oct 30 13:15:53.635709 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.635690    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0297f575-381a-4fed-ad28-6d257c127d64" containerName="revoke-keys"
Oct 30 13:15:53.635817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.635711    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0297f575-381a-4fed-ad28-6d257c127d64" containerName="revoke-keys"
Oct 30 13:15:53.635928 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.635909    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0297f575-381a-4fed-ad28-6d257c127d64" containerName="revoke-keys"
Oct 30 13:15:53.641306 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.641029    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.657546 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.657523    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:15:53.657970 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.657953    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:15:53.673404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.673364    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"]
Oct 30 13:15:53.737923 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.737742    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dpmd\" (UniqueName: \"kubernetes.io/projected/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-kube-api-access-8dpmd\") pod \"maas-api-revoke-sub-0c5315f2-qj5bs\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.738383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.738357    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0c5315f2-qj5bs\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.739711 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.739690    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0c5315f2-qj5bs\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.843311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.842986    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dpmd\" (UniqueName: \"kubernetes.io/projected/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-kube-api-access-8dpmd\") pod \"maas-api-revoke-sub-0c5315f2-qj5bs\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.843311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.843132    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0c5315f2-qj5bs\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.843311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.843187    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0c5315f2-qj5bs\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.844085 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.844053    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0c5315f2-qj5bs\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.847878 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.847855    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0c5315f2-qj5bs\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.883073 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.883041    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dpmd\" (UniqueName: \"kubernetes.io/projected/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-kube-api-access-8dpmd\") pod \"maas-api-revoke-sub-0c5315f2-qj5bs\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:53.965780 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:53.964458    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:54.397951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.397858    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"]
Oct 30 13:15:54.402466 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:54.402430    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf454b33e_bfcc_49bb_b2e9_2cc8ed59a54f.slice/crio-0928e4a9b0c186a293264e9a16da54dacbc9841cb653551db9735dd694d312b5 WatchSource:0}: Error finding container 0928e4a9b0c186a293264e9a16da54dacbc9841cb653551db9735dd694d312b5: Status 404 returned error can't find the container with id 0928e4a9b0c186a293264e9a16da54dacbc9841cb653551db9735dd694d312b5
Oct 30 13:15:54.555305 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.555260    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"]
Oct 30 13:15:54.560804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.560782    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.585448 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.585418    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"]
Oct 30 13:15:54.687940 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.687900    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs" event={"ID":"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f","Type":"ContainerStarted","Data":"0928e4a9b0c186a293264e9a16da54dacbc9841cb653551db9735dd694d312b5"}
Oct 30 13:15:54.694955 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.694926    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8c7f8b52-6x5d9\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.694955 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.694959    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq27r\" (UniqueName: \"kubernetes.io/projected/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-kube-api-access-sq27r\") pod \"maas-api-revoke-sub-8c7f8b52-6x5d9\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.695169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.695120    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8c7f8b52-6x5d9\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.796426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.796395    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8c7f8b52-6x5d9\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.796426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.796424    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq27r\" (UniqueName: \"kubernetes.io/projected/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-kube-api-access-sq27r\") pod \"maas-api-revoke-sub-8c7f8b52-6x5d9\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.796669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.796475    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8c7f8b52-6x5d9\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.797213 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.797191    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8c7f8b52-6x5d9\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.798854 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.798829    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8c7f8b52-6x5d9\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.829982 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.829953    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq27r\" (UniqueName: \"kubernetes.io/projected/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-kube-api-access-sq27r\") pod \"maas-api-revoke-sub-8c7f8b52-6x5d9\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:54.875415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:54.875392    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:55.080438 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:55.080346    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:55.192486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:55.192463    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"]
Oct 30 13:15:55.195082 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:15:55.195048    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652d2429_a2d4_4e03_bc92_bf7b227b8b4c.slice/crio-8cd7f136d05c74233a03cb5e5eee61ad682d4704f1a542bd29874201894cd818 WatchSource:0}: Error finding container 8cd7f136d05c74233a03cb5e5eee61ad682d4704f1a542bd29874201894cd818: Status 404 returned error can't find the container with id 8cd7f136d05c74233a03cb5e5eee61ad682d4704f1a542bd29874201894cd818
Oct 30 13:15:55.696414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:55.696380    5781 generic.go:356] "Generic (PLEG): container finished" podID="f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f" containerID="f59ced293b556cc8950b111af5f48569839ce7584526e12ce85bec074ceb4eb9" exitCode=0
Oct 30 13:15:55.696835 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:55.696514    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs" event={"ID":"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f","Type":"ContainerDied","Data":"f59ced293b556cc8950b111af5f48569839ce7584526e12ce85bec074ceb4eb9"}
Oct 30 13:15:55.698469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:55.698433    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9" event={"ID":"652d2429-a2d4-4e03-bc92-bf7b227b8b4c","Type":"ContainerStarted","Data":"f6fdf15d3f7f0e34e53cf1a9162735ed877ec827fbd0dfe64e698b865efecfdb"}
Oct 30 13:15:55.698601 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:55.698492    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9" event={"ID":"652d2429-a2d4-4e03-bc92-bf7b227b8b4c","Type":"ContainerStarted","Data":"8cd7f136d05c74233a03cb5e5eee61ad682d4704f1a542bd29874201894cd818"}
Oct 30 13:15:55.786558 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:55.786506    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9" podStartSLOduration=1.786494639 podStartE2EDuration="1.786494639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:15:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:15:55.785555675 +0000 UTC m=+1093.082773217" watchObservedRunningTime="2026-10-30 13:15:55.786494639 +0000 UTC m=+1093.083712183"
Oct 30 13:15:56.707041 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.707000    5781 generic.go:356] "Generic (PLEG): container finished" podID="652d2429-a2d4-4e03-bc92-bf7b227b8b4c" containerID="f6fdf15d3f7f0e34e53cf1a9162735ed877ec827fbd0dfe64e698b865efecfdb" exitCode=0
Oct 30 13:15:56.707617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.707084    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9" event={"ID":"652d2429-a2d4-4e03-bc92-bf7b227b8b4c","Type":"ContainerDied","Data":"f6fdf15d3f7f0e34e53cf1a9162735ed877ec827fbd0dfe64e698b865efecfdb"}
Oct 30 13:15:56.709733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.709701    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:56.859482 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.859459    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:56.921857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.921823    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-xks-ca\") pod \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") "
Oct 30 13:15:56.922049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.922001    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-kube-api-access-8dpmd\" (UniqueName: \"kubernetes.io/projected/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-kube-api-access-8dpmd\") pod \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") "
Oct 30 13:15:56.922168 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.922149    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-openshift-ca\") pod \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\" (UID: \"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f\") "
Oct 30 13:15:56.925392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.922548    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-openshift-ca" pod "f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f" (UID: "f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:56.925392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.923237    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:56.925392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.924499    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-kube-api-access-8dpmd" pod "f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f" (UID: "f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f"). InnerVolumeSpecName "kube-api-access-8dpmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:56.926633 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:56.926601    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-xks-ca" pod "f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f" (UID: "f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:57.024871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.024810    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:57.024871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.024834    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dpmd\" (UniqueName: \"kubernetes.io/projected/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f-kube-api-access-8dpmd\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:57.714375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.714349    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"
Oct 30 13:15:57.714375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.714344    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs" event={"ID":"f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f","Type":"ContainerDied","Data":"0928e4a9b0c186a293264e9a16da54dacbc9841cb653551db9735dd694d312b5"}
Oct 30 13:15:57.714831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.714405    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0928e4a9b0c186a293264e9a16da54dacbc9841cb653551db9735dd694d312b5"
Oct 30 13:15:57.846902 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.846878    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:57.935333 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.935302    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-openshift-ca\") pod \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") "
Oct 30 13:15:57.935481 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.935455    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-kube-api-access-sq27r\" (UniqueName: \"kubernetes.io/projected/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-kube-api-access-sq27r\") pod \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") "
Oct 30 13:15:57.935558 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.935499    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-xks-ca\") pod \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\" (UID: \"652d2429-a2d4-4e03-bc92-bf7b227b8b4c\") "
Oct 30 13:15:57.935750 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.935723    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-openshift-ca" pod "652d2429-a2d4-4e03-bc92-bf7b227b8b4c" (UID: "652d2429-a2d4-4e03-bc92-bf7b227b8b4c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:15:57.936112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.936084    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:57.938059 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.938040    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-xks-ca" pod "652d2429-a2d4-4e03-bc92-bf7b227b8b4c" (UID: "652d2429-a2d4-4e03-bc92-bf7b227b8b4c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:15:57.938059 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:57.938043    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-kube-api-access-sq27r" pod "652d2429-a2d4-4e03-bc92-bf7b227b8b4c" (UID: "652d2429-a2d4-4e03-bc92-bf7b227b8b4c"). InnerVolumeSpecName "kube-api-access-sq27r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:15:58.037187 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:58.037126    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sq27r\" (UniqueName: \"kubernetes.io/projected/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-kube-api-access-sq27r\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:58.037187 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:58.037149    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/652d2429-a2d4-4e03-bc92-bf7b227b8b4c-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:15:58.289037 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:58.288960    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:15:58.721682 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:58.721648    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9" event={"ID":"652d2429-a2d4-4e03-bc92-bf7b227b8b4c","Type":"ContainerDied","Data":"8cd7f136d05c74233a03cb5e5eee61ad682d4704f1a542bd29874201894cd818"}
Oct 30 13:15:58.721682 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:58.721667    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"
Oct 30 13:15:58.722196 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:15:58.721693    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cd7f136d05c74233a03cb5e5eee61ad682d4704f1a542bd29874201894cd818"
Oct 30 13:16:00.002908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:00.002871    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:01.002773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:01.002737    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:01.814201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:01.814115    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:01.996454 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:16:01.996419    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:16:02.142527 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.142452    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:02.297494 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.297462    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:02.483591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.483556    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"]
Oct 30 13:16:02.484212 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.484191    5781 cpu_manager.go:414] "containerMap: removing container" podUID="f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f" containerName="revoke-keys"
Oct 30 13:16:02.484335 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.484212    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f" containerName="revoke-keys"
Oct 30 13:16:02.484335 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.484270    5781 cpu_manager.go:414] "containerMap: removing container" podUID="652d2429-a2d4-4e03-bc92-bf7b227b8b4c" containerName="revoke-keys"
Oct 30 13:16:02.484335 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.484278    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="652d2429-a2d4-4e03-bc92-bf7b227b8b4c" containerName="revoke-keys"
Oct 30 13:16:02.484495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.484438    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f" containerName="revoke-keys"
Oct 30 13:16:02.484495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.484459    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="652d2429-a2d4-4e03-bc92-bf7b227b8b4c" containerName="revoke-keys"
Oct 30 13:16:02.488265 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.488244    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.514174 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.514153    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:16:02.514435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.514404    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:16:02.518077 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.518058    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"]
Oct 30 13:16:02.592569 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.592536    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1cf5c368-kfw42\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.592755 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.592736    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbxkp\" (UniqueName: \"kubernetes.io/projected/0b9a4228-15a7-4d39-bc2f-767622117ce9-kube-api-access-tbxkp\") pod \"maas-api-revoke-sub-1cf5c368-kfw42\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.592827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.592811    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1cf5c368-kfw42\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.693618 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.693586    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbxkp\" (UniqueName: \"kubernetes.io/projected/0b9a4228-15a7-4d39-bc2f-767622117ce9-kube-api-access-tbxkp\") pod \"maas-api-revoke-sub-1cf5c368-kfw42\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.693785 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.693644    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1cf5c368-kfw42\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.693785 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.693699    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1cf5c368-kfw42\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.694440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.694413    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1cf5c368-kfw42\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.696004 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.695981    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1cf5c368-kfw42\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.718348 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.718323    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbxkp\" (UniqueName: \"kubernetes.io/projected/0b9a4228-15a7-4d39-bc2f-767622117ce9-kube-api-access-tbxkp\") pod \"maas-api-revoke-sub-1cf5c368-kfw42\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:02.801474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:02.801401    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:03.107786 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:03.107544    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"]
Oct 30 13:16:03.110913 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:16:03.110882    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b9a4228_15a7_4d39_bc2f_767622117ce9.slice/crio-0bde05432033271aaa5e0bce2a778303c651ceaa1857880083844071e1b4489f WatchSource:0}: Error finding container 0bde05432033271aaa5e0bce2a778303c651ceaa1857880083844071e1b4489f: Status 404 returned error can't find the container with id 0bde05432033271aaa5e0bce2a778303c651ceaa1857880083844071e1b4489f
Oct 30 13:16:03.375820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:03.375706    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:03.755824 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:03.755789    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42" event={"ID":"0b9a4228-15a7-4d39-bc2f-767622117ce9","Type":"ContainerStarted","Data":"dce3d4fadca66f127beb0e31704b5fe30ff4b884427a40a8419b16072de04b45"}
Oct 30 13:16:03.755988 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:03.755868    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42" event={"ID":"0b9a4228-15a7-4d39-bc2f-767622117ce9","Type":"ContainerStarted","Data":"0bde05432033271aaa5e0bce2a778303c651ceaa1857880083844071e1b4489f"}
Oct 30 13:16:03.802202 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:03.802154    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42" podStartSLOduration=1.802141371 podStartE2EDuration="1.802141371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:03.798255983 +0000 UTC m=+1101.095473523" watchObservedRunningTime="2026-10-30 13:16:03.802141371 +0000 UTC m=+1101.099358912"
Oct 30 13:16:04.700909 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.699843    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"]
Oct 30 13:16:04.744536 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.744495    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"]
Oct 30 13:16:04.744937 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.744865    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:04.907315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.901788    5781 generic.go:356] "Generic (PLEG): container finished" podID="0b9a4228-15a7-4d39-bc2f-767622117ce9" containerID="dce3d4fadca66f127beb0e31704b5fe30ff4b884427a40a8419b16072de04b45" exitCode=0
Oct 30 13:16:04.907315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.901882    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42" event={"ID":"0b9a4228-15a7-4d39-bc2f-767622117ce9","Type":"ContainerDied","Data":"dce3d4fadca66f127beb0e31704b5fe30ff4b884427a40a8419b16072de04b45"}
Oct 30 13:16:04.966609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.965605    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:04.966609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.965665    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:04.966609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.965709    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f952l\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-kube-api-access-f952l\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:04.966609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.965768    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:04.966609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.965809    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:04.966609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.965863    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:04.966609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.965912    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:04.966609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.965958    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:04.966609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:04.966012    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.069046 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.069008    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f952l\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-kube-api-access-f952l\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.069338 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.069318    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.069464 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.069451    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.069599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.069584    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.069740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.069725    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.069874 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.069858    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.070020 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.070004    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.070187 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.070169    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.078329 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.078302    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.078579 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.077210    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.078716 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.077472    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.078808 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.077691    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.078906 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.077898    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.078995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.076758    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.083695 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.083020    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.085824 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.085806    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.104756 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.104697    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.107544 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.107517    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f952l\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-kube-api-access-f952l\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-xzzm8\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.202525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.202481    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:05.422749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.422713    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:05.553414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.553390    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"]
Oct 30 13:16:05.556240 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:16:05.556213    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f0f44c_ee5a_4d0c_91c7_d1d55fd15c48.slice/crio-b7a98787e1c22cb948b261a32a29c96f0ad4a28960d3345c25d0f352a5f8b7a6 WatchSource:0}: Error finding container b7a98787e1c22cb948b261a32a29c96f0ad4a28960d3345c25d0f352a5f8b7a6: Status 404 returned error can't find the container with id b7a98787e1c22cb948b261a32a29c96f0ad4a28960d3345c25d0f352a5f8b7a6
Oct 30 13:16:05.558771 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.558734    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:05.558957 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.558927    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:05.559065 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.558969    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:05.910359 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.910262    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" event={"ID":"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48","Type":"ContainerStarted","Data":"0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318"}
Oct 30 13:16:05.910969 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.910938    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" event={"ID":"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48","Type":"ContainerStarted","Data":"b7a98787e1c22cb948b261a32a29c96f0ad4a28960d3345c25d0f352a5f8b7a6"}
Oct 30 13:16:05.961974 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:05.961909    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podStartSLOduration=1.961888832 podStartE2EDuration="1.961888832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:05.956775356 +0000 UTC m=+1103.253992899" watchObservedRunningTime="2026-10-30 13:16:05.961888832 +0000 UTC m=+1103.259106374"
Oct 30 13:16:06.051683 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.051660    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:06.091596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.091565    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-openshift-ca\") pod \"0b9a4228-15a7-4d39-bc2f-767622117ce9\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") "
Oct 30 13:16:06.091757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.091708    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b9a4228-15a7-4d39-bc2f-767622117ce9-kube-api-access-tbxkp\" (UniqueName: \"kubernetes.io/projected/0b9a4228-15a7-4d39-bc2f-767622117ce9-kube-api-access-tbxkp\") pod \"0b9a4228-15a7-4d39-bc2f-767622117ce9\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") "
Oct 30 13:16:06.091757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.091739    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-xks-ca\") pod \"0b9a4228-15a7-4d39-bc2f-767622117ce9\" (UID: \"0b9a4228-15a7-4d39-bc2f-767622117ce9\") "
Oct 30 13:16:06.091885 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.091860    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-openshift-ca" pod "0b9a4228-15a7-4d39-bc2f-767622117ce9" (UID: "0b9a4228-15a7-4d39-bc2f-767622117ce9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:16:06.092220 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.092184    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:16:06.093985 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.093959    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b9a4228-15a7-4d39-bc2f-767622117ce9-kube-api-access-tbxkp" pod "0b9a4228-15a7-4d39-bc2f-767622117ce9" (UID: "0b9a4228-15a7-4d39-bc2f-767622117ce9"). InnerVolumeSpecName "kube-api-access-tbxkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:16:06.094101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.094042    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-xks-ca" pod "0b9a4228-15a7-4d39-bc2f-767622117ce9" (UID: "0b9a4228-15a7-4d39-bc2f-767622117ce9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:16:06.193225 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.193194    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbxkp\" (UniqueName: \"kubernetes.io/projected/0b9a4228-15a7-4d39-bc2f-767622117ce9-kube-api-access-tbxkp\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:16:06.193225 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.193219    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0b9a4228-15a7-4d39-bc2f-767622117ce9-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:16:06.202770 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.202749    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:06.204189 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.204166    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 13:16:06.204280 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.204214    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 13:16:06.647883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.647806    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"]
Oct 30 13:16:06.648647 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.648630    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0b9a4228-15a7-4d39-bc2f-767622117ce9" containerName="revoke-keys"
Oct 30 13:16:06.648709 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.648650    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b9a4228-15a7-4d39-bc2f-767622117ce9" containerName="revoke-keys"
Oct 30 13:16:06.648817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.648807    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b9a4228-15a7-4d39-bc2f-767622117ce9" containerName="revoke-keys"
Oct 30 13:16:06.653007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.652990    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.687417 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.687392    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"]
Oct 30 13:16:06.700040 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.700011    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-21dae24e-6khm8\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.700172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.700056    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-21dae24e-6khm8\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.700172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.700161    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqsp8\" (UniqueName: \"kubernetes.io/projected/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-kube-api-access-hqsp8\") pod \"maas-api-revoke-sub-21dae24e-6khm8\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.801404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.801368    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-21dae24e-6khm8\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.801614 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.801419    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-21dae24e-6khm8\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.801614 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.801480    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqsp8\" (UniqueName: \"kubernetes.io/projected/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-kube-api-access-hqsp8\") pod \"maas-api-revoke-sub-21dae24e-6khm8\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.802161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.802132    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-21dae24e-6khm8\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.804454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.804425    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-21dae24e-6khm8\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.854201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.854156    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqsp8\" (UniqueName: \"kubernetes.io/projected/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-kube-api-access-hqsp8\") pod \"maas-api-revoke-sub-21dae24e-6khm8\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:06.941310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.922729    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"
Oct 30 13:16:06.941310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.923389    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42" event={"ID":"0b9a4228-15a7-4d39-bc2f-767622117ce9","Type":"ContainerDied","Data":"0bde05432033271aaa5e0bce2a778303c651ceaa1857880083844071e1b4489f"}
Oct 30 13:16:06.941310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.923468    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bde05432033271aaa5e0bce2a778303c651ceaa1857880083844071e1b4489f"
Oct 30 13:16:06.967966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:06.967915    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:07.204078 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:07.203991    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 13:16:07.204078 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:07.204042    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 13:16:07.279419 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:07.278781    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:07.300801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:07.300752    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"]
Oct 30 13:16:07.303782 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:16:07.303753    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01fb9ec6_241e_4c39_a1d5_c940ae7d1ed4.slice/crio-bc00cbda2605bd327de8827f6b6970e30e963dc409606f4d91346cbb477ceac4 WatchSource:0}: Error finding container bc00cbda2605bd327de8827f6b6970e30e963dc409606f4d91346cbb477ceac4: Status 404 returned error can't find the container with id bc00cbda2605bd327de8827f6b6970e30e963dc409606f4d91346cbb477ceac4
Oct 30 13:16:07.948805 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:07.948770    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8" event={"ID":"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4","Type":"ContainerStarted","Data":"82072b1de625df191ae41adc8e131f768a7ecc7e287b14f34972263a73e81589"}
Oct 30 13:16:07.949192 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:07.948827    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8" event={"ID":"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4","Type":"ContainerStarted","Data":"bc00cbda2605bd327de8827f6b6970e30e963dc409606f4d91346cbb477ceac4"}
Oct 30 13:16:08.006799 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:08.006745    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8" podStartSLOduration=2.006731604 podStartE2EDuration="2.006731604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:07.993511203 +0000 UTC m=+1105.290728745" watchObservedRunningTime="2026-10-30 13:16:08.006731604 +0000 UTC m=+1105.303949145"
Oct 30 13:16:08.203081 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:08.202993    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 13:16:08.203081 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:08.203044    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 13:16:08.956259 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:08.956225    5781 generic.go:356] "Generic (PLEG): container finished" podID="01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4" containerID="82072b1de625df191ae41adc8e131f768a7ecc7e287b14f34972263a73e81589" exitCode=0
Oct 30 13:16:08.956662 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:08.956315    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8" event={"ID":"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4","Type":"ContainerDied","Data":"82072b1de625df191ae41adc8e131f768a7ecc7e287b14f34972263a73e81589"}
Oct 30 13:16:09.157571 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:09.157495    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:09.202747 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:09.202720    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 13:16:09.202891 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:09.202771    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 13:16:10.151525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.151260    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:10.210327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.210265    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 13:16:10.210499 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.210359    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 13:16:10.264661 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.264574    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-xks-ca\") pod \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") "
Oct 30 13:16:10.264661 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.264648    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-openshift-ca\") pod \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") "
Oct 30 13:16:10.264899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.264676    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-kube-api-access-hqsp8\" (UniqueName: \"kubernetes.io/projected/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-kube-api-access-hqsp8\") pod \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\" (UID: \"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4\") "
Oct 30 13:16:10.265359 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.265321    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-openshift-ca" pod "01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4" (UID: "01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:16:10.267078 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.267050    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-xks-ca" pod "01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4" (UID: "01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:16:10.267228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.267142    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-kube-api-access-hqsp8" pod "01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4" (UID: "01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4"). InnerVolumeSpecName "kube-api-access-hqsp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:16:10.365840 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.365804    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:16:10.365840 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.365842    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:16:10.366036 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.365852    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqsp8\" (UniqueName: \"kubernetes.io/projected/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4-kube-api-access-hqsp8\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:16:10.667955 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.667859    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:10.970436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.970403    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8" event={"ID":"01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4","Type":"ContainerDied","Data":"bc00cbda2605bd327de8827f6b6970e30e963dc409606f4d91346cbb477ceac4"}
Oct 30 13:16:10.970609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.970458    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc00cbda2605bd327de8827f6b6970e30e963dc409606f4d91346cbb477ceac4"
Oct 30 13:16:10.970609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:10.970509    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"
Oct 30 13:16:11.199175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:11.199142    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:11.203243 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:11.203219    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 13:16:11.203351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:11.203263    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 13:16:12.186083 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:12.186014    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:12.203655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:12.203611    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 13:16:12.204166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:12.203697    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 13:16:13.203304 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:13.203262    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 13:16:13.203500 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:13.203337    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 13:16:13.967777 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:13.967741    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:14.206778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:14.206747    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:14.207067 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:14.207047    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:14.207835 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:14.207818    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:16:15.523848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:15.523785    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:17.045764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:17.045706    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:18.771657 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:18.771585    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:20.433270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.433230    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:20.797270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.797182    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"]
Oct 30 13:16:20.798067 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.798048    5781 cpu_manager.go:414] "containerMap: removing container" podUID="01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4" containerName="revoke-keys"
Oct 30 13:16:20.798116 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.798072    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4" containerName="revoke-keys"
Oct 30 13:16:20.798283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.798271    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4" containerName="revoke-keys"
Oct 30 13:16:20.804162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.804145    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:20.822881 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.822857    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"]
Oct 30 13:16:20.831161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.831133    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:16:20.831257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.831160    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:16:20.930883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.930856    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9bf149d3-dzm8b\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:20.931031 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.930921    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxpt7\" (UniqueName: \"kubernetes.io/projected/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-kube-api-access-rxpt7\") pod \"maas-api-revoke-sub-9bf149d3-dzm8b\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:20.931074 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:20.931030    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9bf149d3-dzm8b\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:21.032509 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.032471    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9bf149d3-dzm8b\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:21.032676 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.032549    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxpt7\" (UniqueName: \"kubernetes.io/projected/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-kube-api-access-rxpt7\") pod \"maas-api-revoke-sub-9bf149d3-dzm8b\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:21.032676 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.032611    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9bf149d3-dzm8b\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:21.033329 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.033304    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9bf149d3-dzm8b\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:21.034685 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.034660    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9bf149d3-dzm8b\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:21.055745 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.055680    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxpt7\" (UniqueName: \"kubernetes.io/projected/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-kube-api-access-rxpt7\") pod \"maas-api-revoke-sub-9bf149d3-dzm8b\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:21.116970 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.116945    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:21.419782 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.419760    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"]
Oct 30 13:16:21.422674 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:16:21.422646    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ddd7f33_80f2_4517_9673_1bf6e028e6b7.slice/crio-ae0c22e3740942a2ce3c87247b2861981ac728d3ae619ed72f302bd2eeddffc1 WatchSource:0}: Error finding container ae0c22e3740942a2ce3c87247b2861981ac728d3ae619ed72f302bd2eeddffc1: Status 404 returned error can't find the container with id ae0c22e3740942a2ce3c87247b2861981ac728d3ae619ed72f302bd2eeddffc1
Oct 30 13:16:21.721210 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.721183    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"]
Oct 30 13:16:21.725411 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.725391    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.755038 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.755013    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"]
Oct 30 13:16:21.844409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.844374    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.844598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.844436    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73e67102-b858-4c96-ba57-e72d7a647fa2-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.844598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.844547    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.844598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.844590    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.844731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.844658    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.844731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.844708    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.844808 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.844732    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.844884 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.844859    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.844976 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.844961    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbzh5\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-kube-api-access-hbzh5\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946199 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946167    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946199 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946201    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946240    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946260    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946319    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946370    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbzh5\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-kube-api-access-hbzh5\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946770 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946579    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946770 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946679    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946770 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946689    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946770 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946762    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73e67102-b858-4c96-ba57-e72d7a647fa2-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946805    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.946986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.946880    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.947146 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.947116    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.947582 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.947560    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73e67102-b858-4c96-ba57-e72d7a647fa2-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.948713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.948695    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.949195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.949180    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.954114 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.954093    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:21.974216 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.974162    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:21.974466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:21.974449    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbzh5\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-kube-api-access-hbzh5\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:22.042337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:22.040528    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:22.072140 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:22.072093    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b" event={"ID":"5ddd7f33-80f2-4517-9673-1bf6e028e6b7","Type":"ContainerStarted","Data":"04b495d6e237e941cb99489277782d0dad781aa278e581d5317f28eaa2d2385c"}
Oct 30 13:16:22.072308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:22.072163    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b" event={"ID":"5ddd7f33-80f2-4517-9673-1bf6e028e6b7","Type":"ContainerStarted","Data":"ae0c22e3740942a2ce3c87247b2861981ac728d3ae619ed72f302bd2eeddffc1"}
Oct 30 13:16:22.108929 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:22.108894    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:22.117054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:22.117013    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b" podStartSLOduration=2.116999008 podStartE2EDuration="2.116999008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:22.116571989 +0000 UTC m=+1119.413789531" watchObservedRunningTime="2026-10-30 13:16:22.116999008 +0000 UTC m=+1119.414216549"
Oct 30 13:16:22.381937 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:22.381852    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"]
Oct 30 13:16:22.399200 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:16:22.399167    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e67102_b858_4c96_ba57_e72d7a647fa2.slice/crio-7ae76cefc9202e991594093ac20f254ae8603120a20db661889eb78699900b7b WatchSource:0}: Error finding container 7ae76cefc9202e991594093ac20f254ae8603120a20db661889eb78699900b7b: Status 404 returned error can't find the container with id 7ae76cefc9202e991594093ac20f254ae8603120a20db661889eb78699900b7b
Oct 30 13:16:22.401645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:22.401615    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:22.401725 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:22.401679    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:22.401725 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:22.401708    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:23.081060 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:23.081024    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" event={"ID":"73e67102-b858-4c96-ba57-e72d7a647fa2","Type":"ContainerStarted","Data":"d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e"}
Oct 30 13:16:23.081533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:23.081076    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" event={"ID":"73e67102-b858-4c96-ba57-e72d7a647fa2","Type":"ContainerStarted","Data":"7ae76cefc9202e991594093ac20f254ae8603120a20db661889eb78699900b7b"}
Oct 30 13:16:23.082508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:23.082482    5781 generic.go:356] "Generic (PLEG): container finished" podID="5ddd7f33-80f2-4517-9673-1bf6e028e6b7" containerID="04b495d6e237e941cb99489277782d0dad781aa278e581d5317f28eaa2d2385c" exitCode=0
Oct 30 13:16:23.082622 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:23.082566    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b" event={"ID":"5ddd7f33-80f2-4517-9673-1bf6e028e6b7","Type":"ContainerDied","Data":"04b495d6e237e941cb99489277782d0dad781aa278e581d5317f28eaa2d2385c"}
Oct 30 13:16:23.141576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:23.141525    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" podStartSLOduration=2.141511392 podStartE2EDuration="2.141511392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:23.138070575 +0000 UTC m=+1120.435288120" watchObservedRunningTime="2026-10-30 13:16:23.141511392 +0000 UTC m=+1120.438728933"
Oct 30 13:16:23.797508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:23.797471    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:24.041111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.041078    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:24.042837 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.042811    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 13:16:24.042989 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.042859    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 13:16:24.274324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.274302    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:24.384361 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.384267    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-kube-api-access-rxpt7\" (UniqueName: \"kubernetes.io/projected/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-kube-api-access-rxpt7\") pod \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") "
Oct 30 13:16:24.384514 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.384409    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-openshift-ca\") pod \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") "
Oct 30 13:16:24.384514 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.384479    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-xks-ca\") pod \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\" (UID: \"5ddd7f33-80f2-4517-9673-1bf6e028e6b7\") "
Oct 30 13:16:24.384809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.384784    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-openshift-ca" pod "5ddd7f33-80f2-4517-9673-1bf6e028e6b7" (UID: "5ddd7f33-80f2-4517-9673-1bf6e028e6b7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:16:24.384959 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.384945    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:16:24.386381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.386354    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-kube-api-access-rxpt7" pod "5ddd7f33-80f2-4517-9673-1bf6e028e6b7" (UID: "5ddd7f33-80f2-4517-9673-1bf6e028e6b7"). InnerVolumeSpecName "kube-api-access-rxpt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:16:24.386600 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.386585    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-xks-ca" pod "5ddd7f33-80f2-4517-9673-1bf6e028e6b7" (UID: "5ddd7f33-80f2-4517-9673-1bf6e028e6b7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:16:24.486728 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.486687    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:16:24.486728 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:24.486721    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxpt7\" (UniqueName: \"kubernetes.io/projected/5ddd7f33-80f2-4517-9673-1bf6e028e6b7-kube-api-access-rxpt7\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:16:25.041565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:25.041528    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 13:16:25.041749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:25.041599    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 13:16:25.098569 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:25.098538    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"
Oct 30 13:16:25.098761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:25.098568    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b" event={"ID":"5ddd7f33-80f2-4517-9673-1bf6e028e6b7","Type":"ContainerDied","Data":"ae0c22e3740942a2ce3c87247b2861981ac728d3ae619ed72f302bd2eeddffc1"}
Oct 30 13:16:25.098761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:25.098629    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae0c22e3740942a2ce3c87247b2861981ac728d3ae619ed72f302bd2eeddffc1"
Oct 30 13:16:25.401393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:25.401359    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:26.042276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:26.042233    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 13:16:26.042474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:26.042332    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 13:16:26.536772 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:26.536732    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:27.041811 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:27.041774    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 13:16:27.041995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:27.041828    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 13:16:27.146217 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:27.146173    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:28.042092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:28.042061    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 13:16:28.042528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:28.042127    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 13:16:28.719153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:28.719116    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:29.042082 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:29.041998    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 13:16:29.042082 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:29.042052    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 13:16:30.045947 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:30.045918    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:30.046331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:30.046313    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:30.047022 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:30.047003    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:16:30.423457 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:30.423428    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:31.112269 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.112239    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:31.170408 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.170366    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"]
Oct 30 13:16:31.171073 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.171057    5781 cpu_manager.go:414] "containerMap: removing container" podUID="5ddd7f33-80f2-4517-9673-1bf6e028e6b7" containerName="revoke-keys"
Oct 30 13:16:31.171127 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.171074    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ddd7f33-80f2-4517-9673-1bf6e028e6b7" containerName="revoke-keys"
Oct 30 13:16:31.171176 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.171168    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ddd7f33-80f2-4517-9673-1bf6e028e6b7" containerName="revoke-keys"
Oct 30 13:16:31.174648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.174630    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.199615 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.199589    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.199731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.199618    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d38948d-2d60-403c-91d0-0b5f1a534e60-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.199731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.199642    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.199820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.199758    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.199820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.199781    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.199880 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.199859    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.199915 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.199901    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2n2c\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-kube-api-access-x2n2c\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.200024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.200004    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.200082 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.200070    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.225659 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.225628    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"]
Oct 30 13:16:31.300970 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.300934    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.300970 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.300970    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d38948d-2d60-403c-91d0-0b5f1a534e60-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.300999    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301041    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301065    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301116    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301168    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2n2c\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-kube-api-access-x2n2c\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301510 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301236    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301510 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301319    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301693 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301670    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301823 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301800    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301748    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d38948d-2d60-403c-91d0-0b5f1a534e60-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301832    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.301899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.301735    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.303499 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.303480    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.303656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.303635    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.325964 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.325937    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.328909 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.328887    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2n2c\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-kube-api-access-x2n2c\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.488148 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.488114    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:31.866048 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.866025    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"]
Oct 30 13:16:31.868713 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:16:31.868679    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d38948d_2d60_403c_91d0_0b5f1a534e60.slice/crio-3682dd3dd9cc9c84cc2cf70e1b6786595c7ef91b5398ae8f1d80aef8bdaebdc4 WatchSource:0}: Error finding container 3682dd3dd9cc9c84cc2cf70e1b6786595c7ef91b5398ae8f1d80aef8bdaebdc4: Status 404 returned error can't find the container with id 3682dd3dd9cc9c84cc2cf70e1b6786595c7ef91b5398ae8f1d80aef8bdaebdc4
Oct 30 13:16:31.871133 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.871102    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:31.871210 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.871166    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:31.871210 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:31.871196    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:32.000211 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:16:32.000173    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:16:32.190413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:32.190375    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" event={"ID":"2d38948d-2d60-403c-91d0-0b5f1a534e60","Type":"ContainerStarted","Data":"470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1"}
Oct 30 13:16:32.190908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:32.190452    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" event={"ID":"2d38948d-2d60-403c-91d0-0b5f1a534e60","Type":"ContainerStarted","Data":"3682dd3dd9cc9c84cc2cf70e1b6786595c7ef91b5398ae8f1d80aef8bdaebdc4"}
Oct 30 13:16:32.251577 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:32.251518    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podStartSLOduration=1.2515027349999999 podStartE2EDuration="1.251502735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:32.248842214 +0000 UTC m=+1129.546059754" watchObservedRunningTime="2026-10-30 13:16:32.251502735 +0000 UTC m=+1129.548720310"
Oct 30 13:16:32.261096 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:32.261071    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:32.488397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:32.488324    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:32.490003 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:32.489975    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 13:16:32.490118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:32.490052    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 13:16:33.488900 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:33.488861    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 13:16:33.489408 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:33.488936    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 13:16:34.089108 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:34.089076    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:34.491722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:34.491686    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 13:16:34.492465 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:34.492424    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 13:16:35.488770 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:35.488732    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 13:16:35.489071 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:35.489041    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 13:16:36.489088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.489049    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 13:16:36.489514 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.489117    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 13:16:36.665614 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.665582    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"]
Oct 30 13:16:36.670430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.670412    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.753274 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.753192    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"]
Oct 30 13:16:36.761062 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.761031    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.761312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.761271    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.761454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.761390    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.761520 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.761507    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.761593 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.761535    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.761593 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.761563    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.761700 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.761664    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.761760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.761717    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.761820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.761774    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfhqm\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-kube-api-access-zfhqm\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.838665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.838632    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:36.863330 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863270    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863370    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863417    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfhqm\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-kube-api-access-zfhqm\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863444    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863670 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863488    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863670 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863526    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863670 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863610    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863670 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863655    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863861 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863707    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863861 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863740    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.863934 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863918    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.864009 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.863986    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.864119 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.864103    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.864406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.864387    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.866209 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.866189    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.866423 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.866409    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.895992 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.895961    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.909327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.909267    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfhqm\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-kube-api-access-zfhqm\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:36.986008 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:36.985974    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:37.367846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:37.367823    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"]
Oct 30 13:16:37.370046 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:16:37.370017    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596788d7_8cb7_43bd_a7ad_65ca6daf0a64.slice/crio-a2f2eaed04a07f4016a0c2b865923c5247da468d8e07660a59aecf7083c4fdb4 WatchSource:0}: Error finding container a2f2eaed04a07f4016a0c2b865923c5247da468d8e07660a59aecf7083c4fdb4: Status 404 returned error can't find the container with id a2f2eaed04a07f4016a0c2b865923c5247da468d8e07660a59aecf7083c4fdb4
Oct 30 13:16:37.372585 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:37.372551    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:37.372665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:37.372638    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:37.372703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:37.372680    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:16:37.489300 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:37.489268    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 13:16:37.489637 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:37.489328    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 13:16:38.273656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:38.273584    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" event={"ID":"596788d7-8cb7-43bd-a7ad-65ca6daf0a64","Type":"ContainerStarted","Data":"f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9"}
Oct 30 13:16:38.273656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:38.273642    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" event={"ID":"596788d7-8cb7-43bd-a7ad-65ca6daf0a64","Type":"ContainerStarted","Data":"a2f2eaed04a07f4016a0c2b865923c5247da468d8e07660a59aecf7083c4fdb4"}
Oct 30 13:16:38.333574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:38.333520    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" podStartSLOduration=2.333505698 podStartE2EDuration="2.333505698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:16:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:16:38.330235607 +0000 UTC m=+1135.627453154" watchObservedRunningTime="2026-10-30 13:16:38.333505698 +0000 UTC m=+1135.630723240"
Oct 30 13:16:38.489283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:38.489241    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 13:16:38.489451 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:38.489327    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 13:16:38.766375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:38.766338    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:38.986622 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:38.986584    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:38.988411 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:38.988366    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:16:38.988551 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:38.988418    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 13:16:39.203633 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:39.203601    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:39.492999 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:39.492930    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:39.493375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:39.493227    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:39.493923 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:39.493905    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:16:39.986521 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:39.986490    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:16:39.986693 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:39.986553    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 13:16:40.362385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:40.359628    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:40.987120 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:40.987087    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:16:40.987538 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:40.987142    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 13:16:41.986559 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:41.986517    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:16:41.986736 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:41.986573    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 13:16:42.057805 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:42.057764    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:42.336365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:42.336174    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:42.986650 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:42.986618    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:16:42.986813 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:42.986668    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 13:16:43.707830 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:43.707783    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:43.987431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:43.987342    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 13:16:43.987431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:43.987411    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 13:16:45.040604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:45.040570    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:45.041413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:45.041383    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:45.042435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:45.042417    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:16:45.258648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:45.258616    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:45.900094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:45.900059    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:47.052109 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:47.052074    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:48.641766 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:48.641724    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:50.176147 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:50.176110    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:52.078511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:52.078478    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:53.706528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:53.706496    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:55.264871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:55.264788    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:56.891511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:56.891469    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:57.818023 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:57.817994    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:16:59.079145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:16:59.079102    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:00.668410 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:00.668376    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:01.996900 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:17:01.996867    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:17:02.258416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:02.258314    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:04.147261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:04.147226    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:05.526403 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:05.526339    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:06.416842 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.416809    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"]
Oct 30 13:17:06.428012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.427979    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.459576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.459545    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"]
Oct 30 13:17:06.508585 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.508554    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.508746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.508589    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.508746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.508631    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.508863 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.508747    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.508863 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.508795    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.508935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.508906    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.508971 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.508937    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.508971 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.508953    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df5jl\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-kube-api-access-df5jl\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.509129 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.509114    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.610171 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610138    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.610717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610229    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.610717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610250    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.610717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610279    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.610717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610339    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.610717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610370    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.610717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610414    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.610717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610431    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.610717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610457    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df5jl\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-kube-api-access-df5jl\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.611154 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610746    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.611154 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.610927    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.611269 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.611185    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.611269 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.611213    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.611414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.611364    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.612571 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.612553    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.613548 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.613531    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.638772 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.638743    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.638892 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.638863    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-df5jl\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-kube-api-access-df5jl\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:06.742427 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:06.742388    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:07.091501 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:07.091459    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"]
Oct 30 13:17:07.094051 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:17:07.094028    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ebb19b0_d8c8_4f17_aa13_78f210c82265.slice/crio-b6acf0475e11ecd2a0e2e4712d54d2907a2b09806ad0abc688ba2bd34f6ba7b8 WatchSource:0}: Error finding container b6acf0475e11ecd2a0e2e4712d54d2907a2b09806ad0abc688ba2bd34f6ba7b8: Status 404 returned error can't find the container with id b6acf0475e11ecd2a0e2e4712d54d2907a2b09806ad0abc688ba2bd34f6ba7b8
Oct 30 13:17:07.097000 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:07.096969    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:07.097093 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:07.097050    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:07.097093 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:07.097082    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:07.244667 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:07.244630    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:07.506563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:07.506524    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" event={"ID":"9ebb19b0-d8c8-4f17-aa13-78f210c82265","Type":"ContainerStarted","Data":"5a9ec9cb66a3ff780d31660c50cd629cfa2d897bc1ba60479bdbcbd98af0e482"}
Oct 30 13:17:07.506769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:07.506572    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" event={"ID":"9ebb19b0-d8c8-4f17-aa13-78f210c82265","Type":"ContainerStarted","Data":"b6acf0475e11ecd2a0e2e4712d54d2907a2b09806ad0abc688ba2bd34f6ba7b8"}
Oct 30 13:17:07.551083 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:07.551035    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" podStartSLOduration=1.551021905 podStartE2EDuration="1.551021905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:07.550073962 +0000 UTC m=+1164.847291506" watchObservedRunningTime="2026-10-30 13:17:07.551021905 +0000 UTC m=+1164.848239447"
Oct 30 13:17:07.742855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:07.742830    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:08.744372 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:08.744339    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 13:17:08.744768 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:08.744397    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 13:17:08.850385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:08.842136    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:09.743469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:09.743432    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 13:17:09.743774 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:09.743495    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 13:17:10.446904 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:10.446866    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:10.743488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:10.743391    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 13:17:10.743488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:10.743454    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 13:17:11.040678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.040578    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"]
Oct 30 13:17:11.045192 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.045166    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.056715 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.056687    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"]
Oct 30 13:17:11.061656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.061632    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:17:11.061986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.061967    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:17:11.068797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.068766    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjhbv\" (UniqueName: \"kubernetes.io/projected/1d5ceb6f-ff43-4a28-a15a-560beef6112c-kube-api-access-qjhbv\") pod \"maas-api-revoke-sub-b226dff0-rb6p7\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.069043 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.069022    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b226dff0-rb6p7\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.069181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.069134    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b226dff0-rb6p7\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.170117 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.170083    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjhbv\" (UniqueName: \"kubernetes.io/projected/1d5ceb6f-ff43-4a28-a15a-560beef6112c-kube-api-access-qjhbv\") pod \"maas-api-revoke-sub-b226dff0-rb6p7\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.170355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.170153    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b226dff0-rb6p7\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.170355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.170203    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b226dff0-rb6p7\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.170944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.170922    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b226dff0-rb6p7\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.172892 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.172869    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b226dff0-rb6p7\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.194426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.194398    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjhbv\" (UniqueName: \"kubernetes.io/projected/1d5ceb6f-ff43-4a28-a15a-560beef6112c-kube-api-access-qjhbv\") pod \"maas-api-revoke-sub-b226dff0-rb6p7\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.364766 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.364676    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:11.702276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.702248    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"]
Oct 30 13:17:11.704907 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:17:11.704878    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d5ceb6f_ff43_4a28_a15a_560beef6112c.slice/crio-7ec6e9edf3314c041efd0b155a1faf80147a57fd040a0db5d41701e56917b1bf WatchSource:0}: Error finding container 7ec6e9edf3314c041efd0b155a1faf80147a57fd040a0db5d41701e56917b1bf: Status 404 returned error can't find the container with id 7ec6e9edf3314c041efd0b155a1faf80147a57fd040a0db5d41701e56917b1bf
Oct 30 13:17:11.743345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.743205    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 13:17:11.743345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:11.743264    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 13:17:12.353949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:12.353612    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:12.565964 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:12.565923    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7" event={"ID":"1d5ceb6f-ff43-4a28-a15a-560beef6112c","Type":"ContainerStarted","Data":"d100b2324988ea600e867cfaec50c7ba7e68fa0dbf344fe7825f4a8000f02822"}
Oct 30 13:17:12.566166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:12.565982    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7" event={"ID":"1d5ceb6f-ff43-4a28-a15a-560beef6112c","Type":"ContainerStarted","Data":"7ec6e9edf3314c041efd0b155a1faf80147a57fd040a0db5d41701e56917b1bf"}
Oct 30 13:17:12.629552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:12.614996    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7" podStartSLOduration=2.6149756870000003 podStartE2EDuration="2.614975687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:12.610630047 +0000 UTC m=+1169.907847592" watchObservedRunningTime="2026-10-30 13:17:12.614975687 +0000 UTC m=+1169.912193232"
Oct 30 13:17:12.750323 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:12.748740    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 13:17:12.750323 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:12.748812    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 13:17:13.574975 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:13.574942    5781 generic.go:356] "Generic (PLEG): container finished" podID="1d5ceb6f-ff43-4a28-a15a-560beef6112c" containerID="d100b2324988ea600e867cfaec50c7ba7e68fa0dbf344fe7825f4a8000f02822" exitCode=0
Oct 30 13:17:13.575147 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:13.574999    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7" event={"ID":"1d5ceb6f-ff43-4a28-a15a-560beef6112c","Type":"ContainerDied","Data":"d100b2324988ea600e867cfaec50c7ba7e68fa0dbf344fe7825f4a8000f02822"}
Oct 30 13:17:13.746569 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:13.745091    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 13:17:13.746569 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:13.745203    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 13:17:14.222918 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.222879    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:14.741789 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.741764    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:14.748780 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.748754    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:14.749043 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.749025    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:14.749724 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.749710    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:17:14.849993 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.848223    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-xks-ca\") pod \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") "
Oct 30 13:17:14.849993 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.848419    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-openshift-ca\") pod \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") "
Oct 30 13:17:14.849993 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.848525    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d5ceb6f-ff43-4a28-a15a-560beef6112c-kube-api-access-qjhbv\" (UniqueName: \"kubernetes.io/projected/1d5ceb6f-ff43-4a28-a15a-560beef6112c-kube-api-access-qjhbv\") pod \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\" (UID: \"1d5ceb6f-ff43-4a28-a15a-560beef6112c\") "
Oct 30 13:17:14.850678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.850410    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-openshift-ca" pod "1d5ceb6f-ff43-4a28-a15a-560beef6112c" (UID: "1d5ceb6f-ff43-4a28-a15a-560beef6112c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:17:14.853440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.853173    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d5ceb6f-ff43-4a28-a15a-560beef6112c-kube-api-access-qjhbv" pod "1d5ceb6f-ff43-4a28-a15a-560beef6112c" (UID: "1d5ceb6f-ff43-4a28-a15a-560beef6112c"). InnerVolumeSpecName "kube-api-access-qjhbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:17:14.863028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.862975    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-xks-ca" pod "1d5ceb6f-ff43-4a28-a15a-560beef6112c" (UID: "1d5ceb6f-ff43-4a28-a15a-560beef6112c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:17:14.951026 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.950993    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjhbv\" (UniqueName: \"kubernetes.io/projected/1d5ceb6f-ff43-4a28-a15a-560beef6112c-kube-api-access-qjhbv\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:17:14.951026 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.951025    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:17:14.951228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:14.951039    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1d5ceb6f-ff43-4a28-a15a-560beef6112c-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:17:15.592228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.592198    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"
Oct 30 13:17:15.592772 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.592228    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7" event={"ID":"1d5ceb6f-ff43-4a28-a15a-560beef6112c","Type":"ContainerDied","Data":"7ec6e9edf3314c041efd0b155a1faf80147a57fd040a0db5d41701e56917b1bf"}
Oct 30 13:17:15.592772 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.592308    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ec6e9edf3314c041efd0b155a1faf80147a57fd040a0db5d41701e56917b1bf"
Oct 30 13:17:15.788948 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.788915    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"]
Oct 30 13:17:15.789655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.789634    5781 cpu_manager.go:414] "containerMap: removing container" podUID="1d5ceb6f-ff43-4a28-a15a-560beef6112c" containerName="revoke-keys"
Oct 30 13:17:15.789655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.789656    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d5ceb6f-ff43-4a28-a15a-560beef6112c" containerName="revoke-keys"
Oct 30 13:17:15.789793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.789774    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d5ceb6f-ff43-4a28-a15a-560beef6112c" containerName="revoke-keys"
Oct 30 13:17:15.793424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.793397    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.810470 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.810438    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-f8a55e\"/\"distinct-2-w5-f8a55e-kserve-self-signed-certs\""
Oct 30 13:17:15.826219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.826185    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"]
Oct 30 13:17:15.863220 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.863135    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-tmp-dir\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.863416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.863278    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b724a211-f022-417f-aab0-2f78bab05fbd-tls-certs\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.863416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.863405    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w595\" (UniqueName: \"kubernetes.io/projected/b724a211-f022-417f-aab0-2f78bab05fbd-kube-api-access-5w595\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.863514 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.863454    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-model-cache\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.863564 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.863515    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-home\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.863564 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.863544    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-dshm\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.863795 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.863756    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:15.965576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.965538    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w595\" (UniqueName: \"kubernetes.io/projected/b724a211-f022-417f-aab0-2f78bab05fbd-kube-api-access-5w595\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.965771 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.965692    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-model-cache\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.965771 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.965726    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-home\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.965771 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.965758    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-dshm\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.965941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.965918    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-tmp-dir\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.966075 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.966053    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b724a211-f022-417f-aab0-2f78bab05fbd-tls-certs\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.966193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.966170    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-model-cache\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.966193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.966185    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-home\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.966348 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.966275    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-tmp-dir\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.968080 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.968050    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-dshm\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.968407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.968390    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b724a211-f022-417f-aab0-2f78bab05fbd-tls-certs\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:15.996769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:15.996735    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w595\" (UniqueName: \"kubernetes.io/projected/b724a211-f022-417f-aab0-2f78bab05fbd-kube-api-access-5w595\") pod \"distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") " pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:16.055981 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:16.055944    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:16.112621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:16.112587    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:16.426968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:16.426942    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"]
Oct 30 13:17:16.429575 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:17:16.429549    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb724a211_f022_417f_aab0_2f78bab05fbd.slice/crio-46fda9eafbfe7b4a9cd321d489434c37b9500842424ff6bdb6c6b608ac61a004 WatchSource:0}: Error finding container 46fda9eafbfe7b4a9cd321d489434c37b9500842424ff6bdb6c6b608ac61a004: Status 404 returned error can't find the container with id 46fda9eafbfe7b4a9cd321d489434c37b9500842424ff6bdb6c6b608ac61a004
Oct 30 13:17:16.600651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:16.600611    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" event={"ID":"b724a211-f022-417f-aab0-2f78bab05fbd","Type":"ContainerStarted","Data":"7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81"}
Oct 30 13:17:16.601092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:16.600666    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" event={"ID":"b724a211-f022-417f-aab0-2f78bab05fbd","Type":"ContainerStarted","Data":"46fda9eafbfe7b4a9cd321d489434c37b9500842424ff6bdb6c6b608ac61a004"}
Oct 30 13:17:16.643283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:16.643236    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podStartSLOduration=1.643221777 podStartE2EDuration="1.643221777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:16.641098041 +0000 UTC m=+1173.938315586" watchObservedRunningTime="2026-10-30 13:17:16.643221777 +0000 UTC m=+1173.940439319"
Oct 30 13:17:17.113195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:17.113164    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:17.481077 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:17.481033    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:19.204309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:19.204059    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:20.508701 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:20.508646    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:21.190472 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.190429    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"]
Oct 30 13:17:21.199096 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.198614    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.298922 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.298872    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:21.304425 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.304389    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"]
Oct 30 13:17:21.305375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.300506    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.305663 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.305640    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.305817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.305800    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.305988 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.305972    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8211606-b218-40b9-aea3-c6d96034ba89-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.306159 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.306142    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6bh9\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-kube-api-access-h6bh9\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.306348 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.306331    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.306552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.306504    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.306676 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.306660    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.306812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.306799    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8211606-b218-40b9-aea3-c6d96034ba89-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.359932 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.359895    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"]
Oct 30 13:17:21.360322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.360303    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"]
Oct 30 13:17:21.360512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.360217    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.408185 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408157    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.408587 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408527    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.408718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408616    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqv9p\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-kube-api-access-hqv9p\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.408718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408706    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.408828 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408792    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.408885 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408837    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6bh9\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-kube-api-access-h6bh9\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.408938 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408884    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.408938 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408922    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8211606-b218-40b9-aea3-c6d96034ba89-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.409030 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408957    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.409030 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.408996    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.409030 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.409021    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.409179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.409062    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.409179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.409097    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.409179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.409124    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.409179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.409164    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8211606-b218-40b9-aea3-c6d96034ba89-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.409405 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.409232    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.409405 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.409258    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.409405 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.409308    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.411668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.410281    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.411668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.410894    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.411668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.411561    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.411902 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.411805    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.414110 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.412441    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8211606-b218-40b9-aea3-c6d96034ba89-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.417260 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.417233    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8211606-b218-40b9-aea3-c6d96034ba89-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.417565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.417543    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.434755 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.434721    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.440763 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.440689    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6bh9\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-kube-api-access-h6bh9\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.510236 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510194    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.510731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510331    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.510731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510372    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.510731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510397    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.510731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510515    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.510731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510543    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.510731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510576    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.510731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510670    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.510731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510723    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqv9p\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-kube-api-access-hqv9p\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.511147 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510857    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-credential-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.511147 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510896    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-data\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.511147 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.510982    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-certs\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.511147 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.511094    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-socket\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.511147 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.511102    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istiod-ca-cert\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.512836 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.512813    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-envoy\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.513156 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.513136    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-podinfo\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.533592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.533555    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-token\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.533727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.533666    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqv9p\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-kube-api-access-hqv9p\") pod \"e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") " pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.597379 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.597347    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:21.682733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.682704    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:21.925620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.925596    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"]
Oct 30 13:17:21.927337 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:17:21.927308    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8211606_b218_40b9_aea3_c6d96034ba89.slice/crio-5a50e45c487071f6a3eda649525dc974a7f491d43d5e25fca7015e840f017470 WatchSource:0}: Error finding container 5a50e45c487071f6a3eda649525dc974a7f491d43d5e25fca7015e840f017470: Status 404 returned error can't find the container with id 5a50e45c487071f6a3eda649525dc974a7f491d43d5e25fca7015e840f017470
Oct 30 13:17:21.929792 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.929760    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:21.929870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.929824    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:21.929870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:21.929853    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:22.032642 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:22.032408    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"]
Oct 30 13:17:22.035909 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:17:22.035875    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e237b9_8fb9_4fa9_9986_bd2beae17e9b.slice/crio-055b03d33edcd84490f1b5c3230f1d83776effc264587c588ff71fc46183940a WatchSource:0}: Error finding container 055b03d33edcd84490f1b5c3230f1d83776effc264587c588ff71fc46183940a: Status 404 returned error can't find the container with id 055b03d33edcd84490f1b5c3230f1d83776effc264587c588ff71fc46183940a
Oct 30 13:17:22.038787 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:22.038755    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:22.038898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:22.038827    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:22.038898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:22.038885    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:22.773240 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:22.773200    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" event={"ID":"c8211606-b218-40b9-aea3-c6d96034ba89","Type":"ContainerStarted","Data":"7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5"}
Oct 30 13:17:22.773640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:22.773266    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" event={"ID":"c8211606-b218-40b9-aea3-c6d96034ba89","Type":"ContainerStarted","Data":"5a50e45c487071f6a3eda649525dc974a7f491d43d5e25fca7015e840f017470"}
Oct 30 13:17:22.776168 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:22.776137    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" event={"ID":"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b","Type":"ContainerStarted","Data":"a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273"}
Oct 30 13:17:22.776247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:22.776198    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" event={"ID":"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b","Type":"ContainerStarted","Data":"055b03d33edcd84490f1b5c3230f1d83776effc264587c588ff71fc46183940a"}
Oct 30 13:17:22.874549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:22.874496    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" podStartSLOduration=1.874481573 podStartE2EDuration="1.874481573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:22.829928596 +0000 UTC m=+1180.127158735" watchObservedRunningTime="2026-10-30 13:17:22.874481573 +0000 UTC m=+1180.171699116"
Oct 30 13:17:23.125370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:23.119934    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:23.598204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:23.598171    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:23.599851 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:23.599825    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:17:23.599982 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:23.599871    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 13:17:23.683623 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:23.683589    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:23.685368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:23.685346    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 13:17:23.685435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:23.685394    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 13:17:24.598798 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:24.598763    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:17:24.599220 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:24.598818    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 13:17:24.683629 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:24.683593    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 13:17:24.683812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:24.683648    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 13:17:24.899469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:24.899359    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:25.239596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.239521    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" podStartSLOduration=4.239496508 podStartE2EDuration="4.239496508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:22.907611026 +0000 UTC m=+1180.204828570" watchObservedRunningTime="2026-10-30 13:17:25.239496508 +0000 UTC m=+1182.536714052"
Oct 30 13:17:25.247450 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.247409    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"]
Oct 30 13:17:25.261635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.261604    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.274003 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.273562    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"]
Oct 30 13:17:25.277117 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.277089    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:17:25.288802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.288765    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:17:25.337501 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.337457    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ba3b553-mgx57\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.337710 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.337562    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ba3b553-mgx57\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.337772 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.337713    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttk62\" (UniqueName: \"kubernetes.io/projected/57ed8a6b-bfa7-4648-87cc-20e225624628-kube-api-access-ttk62\") pod \"maas-api-revoke-sub-9ba3b553-mgx57\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.440259 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.440219    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ba3b553-mgx57\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.462645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.462606    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttk62\" (UniqueName: \"kubernetes.io/projected/57ed8a6b-bfa7-4648-87cc-20e225624628-kube-api-access-ttk62\") pod \"maas-api-revoke-sub-9ba3b553-mgx57\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.463482 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.463455    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ba3b553-mgx57\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.464244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.441244    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9ba3b553-mgx57\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.466972 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.466949    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9ba3b553-mgx57\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.487819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.487797    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttk62\" (UniqueName: \"kubernetes.io/projected/57ed8a6b-bfa7-4648-87cc-20e225624628-kube-api-access-ttk62\") pod \"maas-api-revoke-sub-9ba3b553-mgx57\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.586358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.585373    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:25.599198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.599111    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:17:25.599198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.599160    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 13:17:25.684221 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.684183    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 13:17:25.684414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.684242    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 13:17:25.968331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:25.967839    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"]
Oct 30 13:17:25.973878 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:17:25.971440    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ed8a6b_bfa7_4648_87cc_20e225624628.slice/crio-1c8cfcf3892afec933159399f9f35e16426ec26a58a0c7d6c6d172814debe10e WatchSource:0}: Error finding container 1c8cfcf3892afec933159399f9f35e16426ec26a58a0c7d6c6d172814debe10e: Status 404 returned error can't find the container with id 1c8cfcf3892afec933159399f9f35e16426ec26a58a0c7d6c6d172814debe10e
Oct 30 13:17:26.129414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.126932    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:26.178461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.178426    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:26.598347 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.598318    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:17:26.598514 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.598364    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 13:17:26.683615 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.683582    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 13:17:26.684097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.683639    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 13:17:26.771309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.771191    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:26.811204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.811154    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57" event={"ID":"57ed8a6b-bfa7-4648-87cc-20e225624628","Type":"ContainerStarted","Data":"aa8ce2bf86690f38abe376a56aafbfecaf6a180e1441f499dd9e014a6dbd00ba"}
Oct 30 13:17:26.811438 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.811220    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57" event={"ID":"57ed8a6b-bfa7-4648-87cc-20e225624628","Type":"ContainerStarted","Data":"1c8cfcf3892afec933159399f9f35e16426ec26a58a0c7d6c6d172814debe10e"}
Oct 30 13:17:26.871106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.871049    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57" podStartSLOduration=1.871034103 podStartE2EDuration="1.871034103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:26.868225587 +0000 UTC m=+1184.165443132" watchObservedRunningTime="2026-10-30 13:17:26.871034103 +0000 UTC m=+1184.168251646"
Oct 30 13:17:26.876358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:26.876335    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:27.598184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:27.598150    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 13:17:27.598402 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:27.598205    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 13:17:27.683505 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:27.683464    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 13:17:27.683668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:27.683535    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 13:17:27.861368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:27.860588    5781 generic.go:356] "Generic (PLEG): container finished" podID="57ed8a6b-bfa7-4648-87cc-20e225624628" containerID="aa8ce2bf86690f38abe376a56aafbfecaf6a180e1441f499dd9e014a6dbd00ba" exitCode=0
Oct 30 13:17:27.861368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:27.861123    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57" event={"ID":"57ed8a6b-bfa7-4648-87cc-20e225624628","Type":"ContainerDied","Data":"aa8ce2bf86690f38abe376a56aafbfecaf6a180e1441f499dd9e014a6dbd00ba"}
Oct 30 13:17:28.574189 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:28.574147    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:28.683532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:28.683497    5781 patch_prober.go:29] interesting pod/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 13:17:28.683702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:28.683568    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 13:17:28.891119 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:28.891087    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:28.892314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:28.892207    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:28.896421 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:28.894782    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:17:29.062023 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.061984    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:29.229154 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.229118    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-xks-ca\") pod \"57ed8a6b-bfa7-4648-87cc-20e225624628\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") "
Oct 30 13:17:29.229365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.229192    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-openshift-ca\") pod \"57ed8a6b-bfa7-4648-87cc-20e225624628\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") "
Oct 30 13:17:29.229415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.229393    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57ed8a6b-bfa7-4648-87cc-20e225624628-kube-api-access-ttk62\" (UniqueName: \"kubernetes.io/projected/57ed8a6b-bfa7-4648-87cc-20e225624628-kube-api-access-ttk62\") pod \"57ed8a6b-bfa7-4648-87cc-20e225624628\" (UID: \"57ed8a6b-bfa7-4648-87cc-20e225624628\") "
Oct 30 13:17:29.229576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.229553    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-openshift-ca" pod "57ed8a6b-bfa7-4648-87cc-20e225624628" (UID: "57ed8a6b-bfa7-4648-87cc-20e225624628"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:17:29.229964 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.229945    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:17:29.231457 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.231432    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-xks-ca" pod "57ed8a6b-bfa7-4648-87cc-20e225624628" (UID: "57ed8a6b-bfa7-4648-87cc-20e225624628"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:17:29.231548 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.231488    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57ed8a6b-bfa7-4648-87cc-20e225624628-kube-api-access-ttk62" pod "57ed8a6b-bfa7-4648-87cc-20e225624628" (UID: "57ed8a6b-bfa7-4648-87cc-20e225624628"). InnerVolumeSpecName "kube-api-access-ttk62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:17:29.331608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.331576    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/57ed8a6b-bfa7-4648-87cc-20e225624628-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:17:29.331608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.331604    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttk62\" (UniqueName: \"kubernetes.io/projected/57ed8a6b-bfa7-4648-87cc-20e225624628-kube-api-access-ttk62\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:17:29.688012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.687981    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:29.688498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.688386    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:29.689074 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.689058    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:17:29.878625 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.878599    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"
Oct 30 13:17:29.878789 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.878592    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57" event={"ID":"57ed8a6b-bfa7-4648-87cc-20e225624628","Type":"ContainerDied","Data":"1c8cfcf3892afec933159399f9f35e16426ec26a58a0c7d6c6d172814debe10e"}
Oct 30 13:17:29.878789 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:29.878745    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c8cfcf3892afec933159399f9f35e16426ec26a58a0c7d6c6d172814debe10e"
Oct 30 13:17:30.122244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:30.122134    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:31.140713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:31.140642    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:31.551539 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:31.551476    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:31.998327 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:17:31.998276    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:17:33.324554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:33.324516    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:34.830415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:34.828540    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:35.002314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:35.001372    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:35.114398 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:35.112391    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:35.114846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:35.114809    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:35.638309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:35.635391    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:35.638309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:35.637898    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:17:35.831761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:35.829003    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:35.831761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:35.829394    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:35.994896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:35.993531    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 13:17:36.002318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:36.002086    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:36.002529 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:36.002514    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:17:36.111313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:36.092855    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:17:36.310323 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:36.310214    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:36.810912 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:36.803863    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:37.792320 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.792274    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"]
Oct 30 13:17:37.792907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.792894    5781 cpu_manager.go:414] "containerMap: removing container" podUID="57ed8a6b-bfa7-4648-87cc-20e225624628" containerName="revoke-keys"
Oct 30 13:17:37.792952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.792909    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57ed8a6b-bfa7-4648-87cc-20e225624628" containerName="revoke-keys"
Oct 30 13:17:37.793047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.793039    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="57ed8a6b-bfa7-4648-87cc-20e225624628" containerName="revoke-keys"
Oct 30 13:17:37.796806 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.796790    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:37.811087 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.811063    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:17:37.822537 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.822509    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:17:37.826582 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.826552    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"]
Oct 30 13:17:37.934596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.934561    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76e0a231-wrvbj\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:37.934760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.934655    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76e0a231-wrvbj\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:37.934972 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:37.934954    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4v5z\" (UniqueName: \"kubernetes.io/projected/dda450ba-2d78-475e-8070-7bf49a1b7e7e-kube-api-access-p4v5z\") pod \"maas-api-revoke-sub-76e0a231-wrvbj\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:38.036058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:38.035997    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4v5z\" (UniqueName: \"kubernetes.io/projected/dda450ba-2d78-475e-8070-7bf49a1b7e7e-kube-api-access-p4v5z\") pod \"maas-api-revoke-sub-76e0a231-wrvbj\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:38.036222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:38.036164    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76e0a231-wrvbj\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:38.036222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:38.036204    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76e0a231-wrvbj\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:38.037010 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:38.036984    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76e0a231-wrvbj\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:38.039087 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:38.039058    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76e0a231-wrvbj\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:38.061100 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:38.061027    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4v5z\" (UniqueName: \"kubernetes.io/projected/dda450ba-2d78-475e-8070-7bf49a1b7e7e-kube-api-access-p4v5z\") pod \"maas-api-revoke-sub-76e0a231-wrvbj\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:38.109972 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:38.109939    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:38.425960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:38.425933    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"]
Oct 30 13:17:38.428730 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:17:38.428695    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda450ba_2d78_475e_8070_7bf49a1b7e7e.slice/crio-738c9ed3131e8ea4f864ddea0b9d200a077ad4ade3d56ea1da35e3882b1eaeb3 WatchSource:0}: Error finding container 738c9ed3131e8ea4f864ddea0b9d200a077ad4ade3d56ea1da35e3882b1eaeb3: Status 404 returned error can't find the container with id 738c9ed3131e8ea4f864ddea0b9d200a077ad4ade3d56ea1da35e3882b1eaeb3
Oct 30 13:17:38.500897 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:38.500864    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:39.066349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:39.066247    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj" event={"ID":"dda450ba-2d78-475e-8070-7bf49a1b7e7e","Type":"ContainerStarted","Data":"45259e454e4104b44b5fc7fd2d4d28b62538da5859ee9bbd77a16d1fdcfee17e"}
Oct 30 13:17:39.066715 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:39.066349    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj" event={"ID":"dda450ba-2d78-475e-8070-7bf49a1b7e7e","Type":"ContainerStarted","Data":"738c9ed3131e8ea4f864ddea0b9d200a077ad4ade3d56ea1da35e3882b1eaeb3"}
Oct 30 13:17:39.125599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:39.125533    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj" podStartSLOduration=2.125515808 podStartE2EDuration="2.125515808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:39.121949775 +0000 UTC m=+1196.419167315" watchObservedRunningTime="2026-10-30 13:17:39.125515808 +0000 UTC m=+1196.422733350"
Oct 30 13:17:40.010636 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:40.010595    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:40.074392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:40.074357    5781 generic.go:356] "Generic (PLEG): container finished" podID="dda450ba-2d78-475e-8070-7bf49a1b7e7e" containerID="45259e454e4104b44b5fc7fd2d4d28b62538da5859ee9bbd77a16d1fdcfee17e" exitCode=0
Oct 30 13:17:40.074786 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:40.074447    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj" event={"ID":"dda450ba-2d78-475e-8070-7bf49a1b7e7e","Type":"ContainerDied","Data":"45259e454e4104b44b5fc7fd2d4d28b62538da5859ee9bbd77a16d1fdcfee17e"}
Oct 30 13:17:41.255034 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.255007    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:41.292690 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.292655    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"]
Oct 30 13:17:41.293376 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.293358    5781 cpu_manager.go:414] "containerMap: removing container" podUID="dda450ba-2d78-475e-8070-7bf49a1b7e7e" containerName="revoke-keys"
Oct 30 13:17:41.293376 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.293377    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dda450ba-2d78-475e-8070-7bf49a1b7e7e" containerName="revoke-keys"
Oct 30 13:17:41.293546 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.293534    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="dda450ba-2d78-475e-8070-7bf49a1b7e7e" containerName="revoke-keys"
Oct 30 13:17:41.298051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.298024    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.319910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.319879    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-031737\"/\"openshift-service-ca.crt\""
Oct 30 13:17:41.320870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.320845    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-031737\"/\"distinct-2-w0-031737-kserve-self-signed-certs\""
Oct 30 13:17:41.321150 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.321135    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-031737\"/\"default-dockercfg-dxfjj\""
Oct 30 13:17:41.325596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.325567    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-031737\"/\"kube-root-ca.crt\""
Oct 30 13:17:41.330099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.330054    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"]
Oct 30 13:17:41.384801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.384707    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dda450ba-2d78-475e-8070-7bf49a1b7e7e-kube-api-access-p4v5z\" (UniqueName: \"kubernetes.io/projected/dda450ba-2d78-475e-8070-7bf49a1b7e7e-kube-api-access-p4v5z\") pod \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") "
Oct 30 13:17:41.384801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.384771    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-openshift-ca\") pod \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") "
Oct 30 13:17:41.385040 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.384946    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-xks-ca\") pod \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\" (UID: \"dda450ba-2d78-475e-8070-7bf49a1b7e7e\") "
Oct 30 13:17:41.385191 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.385159    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-openshift-ca" pod "dda450ba-2d78-475e-8070-7bf49a1b7e7e" (UID: "dda450ba-2d78-475e-8070-7bf49a1b7e7e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:17:41.385413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.385389    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f83e9b-225d-46a0-9976-ebedbc846a72-tls-certs\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.385656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.385616    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbxgd\" (UniqueName: \"kubernetes.io/projected/b3f83e9b-225d-46a0-9976-ebedbc846a72-kube-api-access-zbxgd\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.385785 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.385766    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-tmp-dir\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.385873 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.385851    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-model-cache\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.385960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.385946    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-dshm\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.386064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.386040    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-home\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.386175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.386161    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:17:41.387099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.387077    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-xks-ca" pod "dda450ba-2d78-475e-8070-7bf49a1b7e7e" (UID: "dda450ba-2d78-475e-8070-7bf49a1b7e7e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:17:41.387189 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.387147    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dda450ba-2d78-475e-8070-7bf49a1b7e7e-kube-api-access-p4v5z" pod "dda450ba-2d78-475e-8070-7bf49a1b7e7e" (UID: "dda450ba-2d78-475e-8070-7bf49a1b7e7e"). InnerVolumeSpecName "kube-api-access-p4v5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:17:41.393606 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.393578    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:41.488230 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.488072    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbxgd\" (UniqueName: \"kubernetes.io/projected/b3f83e9b-225d-46a0-9976-ebedbc846a72-kube-api-access-zbxgd\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.488230 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.488183    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-tmp-dir\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.488508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.488258    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-model-cache\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.488508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.488320    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-dshm\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.488508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.488364    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-home\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.488508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.488438    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f83e9b-225d-46a0-9976-ebedbc846a72-tls-certs\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.488727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.488593    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dda450ba-2d78-475e-8070-7bf49a1b7e7e-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:17:41.488727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.488612    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4v5z\" (UniqueName: \"kubernetes.io/projected/dda450ba-2d78-475e-8070-7bf49a1b7e7e-kube-api-access-p4v5z\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:17:41.488727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.488663    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-tmp-dir\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.489237 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.489210    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-model-cache\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.489447 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.489337    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-home\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.491508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.491482    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-dshm\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.491938 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.491914    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f83e9b-225d-46a0-9976-ebedbc846a72-tls-certs\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.526971 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.519399    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbxgd\" (UniqueName: \"kubernetes.io/projected/b3f83e9b-225d-46a0-9976-ebedbc846a72-kube-api-access-zbxgd\") pod \"distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") " pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.626627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.626584    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:41.661084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.661044    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:41.966256 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:41.966225    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"]
Oct 30 13:17:41.968816 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:17:41.968784    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3f83e9b_225d_46a0_9976_ebedbc846a72.slice/crio-25a2d525c4fe5866f1b79fa4a5643bb15f78d210d51b7eb54b6a73ee1d8f893d WatchSource:0}: Error finding container 25a2d525c4fe5866f1b79fa4a5643bb15f78d210d51b7eb54b6a73ee1d8f893d: Status 404 returned error can't find the container with id 25a2d525c4fe5866f1b79fa4a5643bb15f78d210d51b7eb54b6a73ee1d8f893d
Oct 30 13:17:42.090856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:42.090819    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" event={"ID":"b3f83e9b-225d-46a0-9976-ebedbc846a72","Type":"ContainerStarted","Data":"b7b3f497bb06c8e30fa32aa212119682dff2b47997e018f8799556845b9db1ca"}
Oct 30 13:17:42.090856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:42.090869    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" event={"ID":"b3f83e9b-225d-46a0-9976-ebedbc846a72","Type":"ContainerStarted","Data":"25a2d525c4fe5866f1b79fa4a5643bb15f78d210d51b7eb54b6a73ee1d8f893d"}
Oct 30 13:17:42.092355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:42.092332    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"
Oct 30 13:17:42.092355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:42.092339    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj" event={"ID":"dda450ba-2d78-475e-8070-7bf49a1b7e7e","Type":"ContainerDied","Data":"738c9ed3131e8ea4f864ddea0b9d200a077ad4ade3d56ea1da35e3882b1eaeb3"}
Oct 30 13:17:42.092537 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:42.092389    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="738c9ed3131e8ea4f864ddea0b9d200a077ad4ade3d56ea1da35e3882b1eaeb3"
Oct 30 13:17:42.134784 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:42.134721    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podStartSLOduration=1.134704001 podStartE2EDuration="1.134704001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:42.131185727 +0000 UTC m=+1199.428403271" watchObservedRunningTime="2026-10-30 13:17:42.134704001 +0000 UTC m=+1199.431921548"
Oct 30 13:17:42.627198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:42.627156    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:42.991380 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:42.991350    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:17:42.993165 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:42.993140    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:17:43.317079 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:43.316969    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:44.914827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:44.914792    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:46.007579 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:46.007550    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:46.323373 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:46.323271    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:47.808224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:47.808181    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:49.474227 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:49.474188    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:50.980100 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:50.980062    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:51.627913 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:51.627882    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:51.641366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:51.641336    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:52.175678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.175641    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"]
Oct 30 13:17:52.181222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.181200    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:52.181381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.181307    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.232446 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.232413    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"]
Oct 30 13:17:52.242763 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.242727    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.242917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.242867    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.243008 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.242943    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.243068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.243007    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a940310d-3db1-4c9c-bf94-399db4eab1a3-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.243068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.243042    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.243178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.243089    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.243178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.243119    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.243178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.243167    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.243387 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.243317    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgh9c\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-kube-api-access-vgh9c\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345458 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345426    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345633 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345518    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345633 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345543    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345633 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345593    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a940310d-3db1-4c9c-bf94-399db4eab1a3-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345633 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345619    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345825 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345666    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345825 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345698    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345825 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345761    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345929 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345845    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.345929 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.345850    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgh9c\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-kube-api-access-vgh9c\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.346128 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.346104    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-credential-socket\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.346180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.346164    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-data\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.346498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.346473    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-certs\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.346774 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.346756    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a940310d-3db1-4c9c-bf94-399db4eab1a3-istiod-ca-cert\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.348833 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.348800    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-envoy\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.349003 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.348985    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-podinfo\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.371142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.371111    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-token\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.371600 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.371582    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgh9c\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-kube-api-access-vgh9c\") pod \"e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") " pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.496371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.496257    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:52.850167 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.850129    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"]
Oct 30 13:17:52.854178 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:17:52.854125    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda940310d_3db1_4c9c_bf94_399db4eab1a3.slice/crio-71cb0bccf280e324a03a6ccd50bdd4b1bb75d610a6ea60e5abdc588608a5e257 WatchSource:0}: Error finding container 71cb0bccf280e324a03a6ccd50bdd4b1bb75d610a6ea60e5abdc588608a5e257: Status 404 returned error can't find the container with id 71cb0bccf280e324a03a6ccd50bdd4b1bb75d610a6ea60e5abdc588608a5e257
Oct 30 13:17:52.902314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.892936    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:52.902314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.893021    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:52.902314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.893062    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:17:52.912812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:52.909665    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:53.207001 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:53.206876    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" event={"ID":"a940310d-3db1-4c9c-bf94-399db4eab1a3","Type":"ContainerStarted","Data":"1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1"}
Oct 30 13:17:53.207001 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:53.206941    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" event={"ID":"a940310d-3db1-4c9c-bf94-399db4eab1a3","Type":"ContainerStarted","Data":"71cb0bccf280e324a03a6ccd50bdd4b1bb75d610a6ea60e5abdc588608a5e257"}
Oct 30 13:17:53.277337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:53.277259    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podStartSLOduration=1.277239518 podStartE2EDuration="1.277239518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:17:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:17:53.275703336 +0000 UTC m=+1210.572920880" watchObservedRunningTime="2026-10-30 13:17:53.277239518 +0000 UTC m=+1210.574457063"
Oct 30 13:17:53.496641 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:53.496412    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:17:54.501314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:54.501033    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:17:54.501314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:54.501107    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 13:17:54.924388 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:54.916621    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:55.101313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.096468    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:55.102116 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.102075    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:55.103140 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.103109    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:55.184367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.183733    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:17:55.314374 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.314191    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 13:17:55.383322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.360248    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:55.498384 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.497778    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:17:55.498384 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.497878    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 13:17:55.646476 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.646419    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:17:55.923373 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.920386    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 13:17:55.923373 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:55.921134    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:17:56.104312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.099418    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:56.104312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.100034    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": context deadline exceeded"
Oct 30 13:17:56.104312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.100348    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:17:56.104312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.102535    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:56.104312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.102849    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:17:56.104312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.103666    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:56.104312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.103940    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:17:56.197877 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.145685    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:17:56.197877 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.190924    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:17:56.197877 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.191351    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:56.213342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.211856    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:17:56.220850 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.220818    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:17:56.373677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.373635    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 13:17:56.402867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.402825    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:56.415055 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.414241    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:17:56.477439 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.477350    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:17:56.498119 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.498084    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:17:56.498362 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.498157    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 13:17:56.502170 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:56.502146    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:17:57.497034 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:57.497001    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:17:57.497462 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:57.497061    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 13:17:57.546311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:57.536520    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:58.496775 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:58.496735    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:17:58.496987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:58.496804    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 13:17:59.067625 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:59.067587    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:17:59.496946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:59.496915    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:17:59.497172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:17:59.497151    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 13:18:00.496803 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:00.496761    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused" start-of-body=
Oct 30 13:18:00.497235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:00.496835    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": dial tcp 10.134.0.90:15021: connect: connection refused"
Oct 30 13:18:00.910047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:00.909966    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:01.853713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:01.853682    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:18:01.854271 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:01.854200    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:18:01.855364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:01.855330    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:18:02.008139 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:18:02.008089    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:18:02.371699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:02.371660    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:03.918011 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:03.917971    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:05.682907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:05.682829    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:06.653545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.653512    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"]
Oct 30 13:18:06.658150 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.658128    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.684463 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.684432    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.684868 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.684485    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.684868 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.684664    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.684868 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.684724    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56lfx\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-kube-api-access-56lfx\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.684868 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.684755    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.684868 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.684834    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d198200-2f59-46b0-907d-d61d2c6cb378-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.685055 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.684873    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.685055 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.684890    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.685055 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.684990    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.692311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.692269    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"]
Oct 30 13:18:06.786061 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786020    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786082    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56lfx\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-kube-api-access-56lfx\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786103    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786133    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d198200-2f59-46b0-907d-d61d2c6cb378-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786161    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786283 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786177    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786334    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786544    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786605    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786782 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786663    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-data\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786782 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786720    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-certs\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786893 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786773    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.786893 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.786856    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-credential-socket\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.787538 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.787513    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d198200-2f59-46b0-907d-d61d2c6cb378-istiod-ca-cert\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.789104 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.789080    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-envoy\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.789318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.789275    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-podinfo\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.811992 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.811932    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56lfx\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-kube-api-access-56lfx\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.814682 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.814645    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-token\") pod \"e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") " pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:06.976065 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:06.976029    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:07.123501 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:07.123468    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:07.307671 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:07.307641    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"]
Oct 30 13:18:07.309676 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:18:07.309649    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d198200_2f59_46b0_907d_d61d2c6cb378.slice/crio-e88fb2c7fd863496e9f6667c863e10ae760201f823e05ff7642e565bf08fadbe WatchSource:0}: Error finding container e88fb2c7fd863496e9f6667c863e10ae760201f823e05ff7642e565bf08fadbe: Status 404 returned error can't find the container with id e88fb2c7fd863496e9f6667c863e10ae760201f823e05ff7642e565bf08fadbe
Oct 30 13:18:07.312198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:07.312162    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:18:07.312325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:07.312235    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:18:07.312325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:07.312269    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:18:07.567533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:07.567435    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" event={"ID":"6d198200-2f59-46b0-907d-d61d2c6cb378","Type":"ContainerStarted","Data":"1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2"}
Oct 30 13:18:07.567533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:07.567498    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" event={"ID":"6d198200-2f59-46b0-907d-d61d2c6cb378","Type":"ContainerStarted","Data":"e88fb2c7fd863496e9f6667c863e10ae760201f823e05ff7642e565bf08fadbe"}
Oct 30 13:18:07.619439 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:07.619323    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podStartSLOduration=1.619300794 podStartE2EDuration="1.619300794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:07.611769716 +0000 UTC m=+1224.908987261" watchObservedRunningTime="2026-10-30 13:18:07.619300794 +0000 UTC m=+1224.916518334"
Oct 30 13:18:07.976567 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:07.976531    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:08.290539 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.290447    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"]
Oct 30 13:18:08.295114 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.295083    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.325381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.316540    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:08.340420 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.334581    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:08.349241 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.348104    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"]
Oct 30 13:18:08.456314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.453544    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxxst\" (UniqueName: \"kubernetes.io/projected/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-kube-api-access-jxxst\") pod \"maas-api-revoke-sub-1ffa313c-dfvmf\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.456314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.453593    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1ffa313c-dfvmf\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.456314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.453674    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1ffa313c-dfvmf\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.564393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.563965    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxxst\" (UniqueName: \"kubernetes.io/projected/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-kube-api-access-jxxst\") pod \"maas-api-revoke-sub-1ffa313c-dfvmf\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.564393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.564086    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1ffa313c-dfvmf\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.564393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.564190    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1ffa313c-dfvmf\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.568328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.565270    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1ffa313c-dfvmf\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.654310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.653509    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxxst\" (UniqueName: \"kubernetes.io/projected/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-kube-api-access-jxxst\") pod \"maas-api-revoke-sub-1ffa313c-dfvmf\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.678310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.659303    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1ffa313c-dfvmf\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:08.889835 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.881358    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:08.928619 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:08.928590    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:09.034309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:09.033457    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 13:18:09.034309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:09.033541    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 13:18:10.172474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.172428    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 13:18:10.209631 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.209585    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 13:18:10.231481 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.231396    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:10.260370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.232572    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:18:10.368312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.348897    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:18:10.460511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.446389    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 13:18:10.509478 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.491752    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": context deadline exceeded"
Oct 30 13:18:10.556921 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.554575    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"]
Oct 30 13:18:10.556921 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.555483    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:10.645456 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.645417    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:18:10.748313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:10.741476    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:11.004583 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:11.004496    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 13:18:11.005155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:11.004800    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 13:18:11.463065 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:11.463030    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:11.486800 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:11.486764    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf" event={"ID":"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f","Type":"ContainerStarted","Data":"fcc9d30fb8ea2059833338c8096ad72ca47aeb156272008ebc3223a749353c30"}
Oct 30 13:18:11.487038 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:11.487025    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf" event={"ID":"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f","Type":"ContainerStarted","Data":"66ec71bd5d09fe13ed874a1fb4113f9fcf1e6190f10a8d9463e3373ee75ac969"}
Oct 30 13:18:11.976400 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:11.976369    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 13:18:11.976640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:11.976443    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 13:18:12.083727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:12.083331    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:12.264498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:12.264469    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:12.496116 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:12.496033    5781 generic.go:356] "Generic (PLEG): container finished" podID="321ca0c6-0be9-4cf6-bad5-d9b91ed2096f" containerID="fcc9d30fb8ea2059833338c8096ad72ca47aeb156272008ebc3223a749353c30" exitCode=0
Oct 30 13:18:12.496508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:12.496111    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf" event={"ID":"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f","Type":"ContainerDied","Data":"fcc9d30fb8ea2059833338c8096ad72ca47aeb156272008ebc3223a749353c30"}
Oct 30 13:18:12.976630 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:12.976595    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 13:18:12.976794 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:12.976683    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 13:18:13.356177 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.356095    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:13.706139 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.706110    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:13.860060 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.860026    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-kube-api-access-jxxst\" (UniqueName: \"kubernetes.io/projected/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-kube-api-access-jxxst\") pod \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") "
Oct 30 13:18:13.860351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.860275    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-xks-ca\") pod \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") "
Oct 30 13:18:13.860537 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.860520    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-openshift-ca\") pod \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\" (UID: \"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f\") "
Oct 30 13:18:13.860870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.860842    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-openshift-ca" pod "321ca0c6-0be9-4cf6-bad5-d9b91ed2096f" (UID: "321ca0c6-0be9-4cf6-bad5-d9b91ed2096f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:13.861074 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.861058    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:13.862358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.862327    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-xks-ca" pod "321ca0c6-0be9-4cf6-bad5-d9b91ed2096f" (UID: "321ca0c6-0be9-4cf6-bad5-d9b91ed2096f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:13.862451 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.862352    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-kube-api-access-jxxst" pod "321ca0c6-0be9-4cf6-bad5-d9b91ed2096f" (UID: "321ca0c6-0be9-4cf6-bad5-d9b91ed2096f"). InnerVolumeSpecName "kube-api-access-jxxst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:13.962967 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.962934    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxxst\" (UniqueName: \"kubernetes.io/projected/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-kube-api-access-jxxst\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:13.962967 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.962958    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:13.977419 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.977388    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 13:18:13.977591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:13.977454    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 13:18:14.517520 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:14.517486    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"
Oct 30 13:18:14.517520 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:14.517512    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf" event={"ID":"321ca0c6-0be9-4cf6-bad5-d9b91ed2096f","Type":"ContainerDied","Data":"66ec71bd5d09fe13ed874a1fb4113f9fcf1e6190f10a8d9463e3373ee75ac969"}
Oct 30 13:18:14.517798 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:14.517562    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66ec71bd5d09fe13ed874a1fb4113f9fcf1e6190f10a8d9463e3373ee75ac969"
Oct 30 13:18:14.976876 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:14.976834    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused" start-of-body=
Oct 30 13:18:14.977276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:14.976905    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.91:15021/healthz/ready\": dial tcp 10.134.0.91:15021: connect: connection refused"
Oct 30 13:18:15.056830 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:15.056724    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:15.981259 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:15.981231    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:15.981664 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:15.981551    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:15.982185 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:15.982165    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:18:16.502166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:16.502091    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:16.674699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:16.674666    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:18.197736 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:18.197696    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:19.907848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:19.907804    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:20.815329 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.815297    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"]
Oct 30 13:18:20.815961 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.815945    5781 cpu_manager.go:414] "containerMap: removing container" podUID="321ca0c6-0be9-4cf6-bad5-d9b91ed2096f" containerName="revoke-keys"
Oct 30 13:18:20.816020 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.815963    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="321ca0c6-0be9-4cf6-bad5-d9b91ed2096f" containerName="revoke-keys"
Oct 30 13:18:20.816099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.816088    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="321ca0c6-0be9-4cf6-bad5-d9b91ed2096f" containerName="revoke-keys"
Oct 30 13:18:20.820043 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.820027    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:20.845927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.845900    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:20.849172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.849138    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"]
Oct 30 13:18:20.855541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.855520    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:20.865556 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.865524    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6r5h\" (UniqueName: \"kubernetes.io/projected/cf381d3f-d08f-43bd-973b-71c63ba88279-kube-api-access-s6r5h\") pod \"maas-api-revoke-sub-b8e408f6-qsqhp\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:20.865675 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.865644    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8e408f6-qsqhp\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:20.865881 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.865858    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8e408f6-qsqhp\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:20.966987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.966955    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6r5h\" (UniqueName: \"kubernetes.io/projected/cf381d3f-d08f-43bd-973b-71c63ba88279-kube-api-access-s6r5h\") pod \"maas-api-revoke-sub-b8e408f6-qsqhp\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:20.967393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.967010    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8e408f6-qsqhp\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:20.967393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.967088    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8e408f6-qsqhp\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:20.967745 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.967721    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b8e408f6-qsqhp\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:20.969465 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.969437    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b8e408f6-qsqhp\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:20.993718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:20.993693    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6r5h\" (UniqueName: \"kubernetes.io/projected/cf381d3f-d08f-43bd-973b-71c63ba88279-kube-api-access-s6r5h\") pod \"maas-api-revoke-sub-b8e408f6-qsqhp\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:21.134579 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:21.134483    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:21.450988 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:21.450957    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"]
Oct 30 13:18:21.453362 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:18:21.453330    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf381d3f_d08f_43bd_973b_71c63ba88279.slice/crio-ae6010df0b0f3689c657c3361ee721ec13d67ef074ef0bf3a05aa8ed8c00fd97 WatchSource:0}: Error finding container ae6010df0b0f3689c657c3361ee721ec13d67ef074ef0bf3a05aa8ed8c00fd97: Status 404 returned error can't find the container with id ae6010df0b0f3689c657c3361ee721ec13d67ef074ef0bf3a05aa8ed8c00fd97
Oct 30 13:18:21.535867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:21.535819    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:21.586065 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:21.586028    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp" event={"ID":"cf381d3f-d08f-43bd-973b-71c63ba88279","Type":"ContainerStarted","Data":"ae6010df0b0f3689c657c3361ee721ec13d67ef074ef0bf3a05aa8ed8c00fd97"}
Oct 30 13:18:22.595539 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:22.595502    5781 generic.go:356] "Generic (PLEG): container finished" podID="cf381d3f-d08f-43bd-973b-71c63ba88279" containerID="37669ae876e7e7c01f29e196c5a0907b71c92621eab56b07990d91d65310989c" exitCode=0
Oct 30 13:18:22.595951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:22.595572    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp" event={"ID":"cf381d3f-d08f-43bd-973b-71c63ba88279","Type":"ContainerDied","Data":"37669ae876e7e7c01f29e196c5a0907b71c92621eab56b07990d91d65310989c"}
Oct 30 13:18:23.088428 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.088369    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:23.761158 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.761133    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:23.806048 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.806007    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-openshift-ca\") pod \"cf381d3f-d08f-43bd-973b-71c63ba88279\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") "
Oct 30 13:18:23.806193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.806058    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-xks-ca\") pod \"cf381d3f-d08f-43bd-973b-71c63ba88279\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") "
Oct 30 13:18:23.806272 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.806250    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf381d3f-d08f-43bd-973b-71c63ba88279-kube-api-access-s6r5h\" (UniqueName: \"kubernetes.io/projected/cf381d3f-d08f-43bd-973b-71c63ba88279-kube-api-access-s6r5h\") pod \"cf381d3f-d08f-43bd-973b-71c63ba88279\" (UID: \"cf381d3f-d08f-43bd-973b-71c63ba88279\") "
Oct 30 13:18:23.806887 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.806817    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-openshift-ca" pod "cf381d3f-d08f-43bd-973b-71c63ba88279" (UID: "cf381d3f-d08f-43bd-973b-71c63ba88279"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:23.807736 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.807708    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:23.808747 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.808721    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-xks-ca" pod "cf381d3f-d08f-43bd-973b-71c63ba88279" (UID: "cf381d3f-d08f-43bd-973b-71c63ba88279"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:23.809168 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.809145    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf381d3f-d08f-43bd-973b-71c63ba88279-kube-api-access-s6r5h" pod "cf381d3f-d08f-43bd-973b-71c63ba88279" (UID: "cf381d3f-d08f-43bd-973b-71c63ba88279"). InnerVolumeSpecName "kube-api-access-s6r5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:23.909227 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.909135    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cf381d3f-d08f-43bd-973b-71c63ba88279-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:23.909227 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:23.909164    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6r5h\" (UniqueName: \"kubernetes.io/projected/cf381d3f-d08f-43bd-973b-71c63ba88279-kube-api-access-s6r5h\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:24.610482 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:24.610449    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"
Oct 30 13:18:24.610482 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:24.610461    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp" event={"ID":"cf381d3f-d08f-43bd-973b-71c63ba88279","Type":"ContainerDied","Data":"ae6010df0b0f3689c657c3361ee721ec13d67ef074ef0bf3a05aa8ed8c00fd97"}
Oct 30 13:18:24.610698 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:24.610512    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae6010df0b0f3689c657c3361ee721ec13d67ef074ef0bf3a05aa8ed8c00fd97"
Oct 30 13:18:24.714011 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:24.713977    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:26.313827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:26.313778    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:27.899353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:27.899266    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:28.110216 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.110180    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"]
Oct 30 13:18:28.111253 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.111230    5781 cpu_manager.go:414] "containerMap: removing container" podUID="cf381d3f-d08f-43bd-973b-71c63ba88279" containerName="revoke-keys"
Oct 30 13:18:28.111388 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.111256    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf381d3f-d08f-43bd-973b-71c63ba88279" containerName="revoke-keys"
Oct 30 13:18:28.111563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.111540    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf381d3f-d08f-43bd-973b-71c63ba88279" containerName="revoke-keys"
Oct 30 13:18:28.116395 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.116371    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.141762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.141727    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"]
Oct 30 13:18:28.144875 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.144854    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:28.145214 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.145158    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:28.183333 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.179280    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqmhx\" (UniqueName: \"kubernetes.io/projected/c7179a4d-bc50-49c9-b605-27be4224c533-kube-api-access-vqmhx\") pod \"maas-api-revoke-sub-04d0ccf4-flksz\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.183333 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.179547    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-04d0ccf4-flksz\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.183333 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.179596    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-xks-ca\") pod \"maas-api-revoke-sub-04d0ccf4-flksz\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.293312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.292261    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqmhx\" (UniqueName: \"kubernetes.io/projected/c7179a4d-bc50-49c9-b605-27be4224c533-kube-api-access-vqmhx\") pod \"maas-api-revoke-sub-04d0ccf4-flksz\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.293312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.292551    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-04d0ccf4-flksz\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.293312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.292607    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-xks-ca\") pod \"maas-api-revoke-sub-04d0ccf4-flksz\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.305322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.300168    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-04d0ccf4-flksz\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.316837 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.316805    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-xks-ca\") pod \"maas-api-revoke-sub-04d0ccf4-flksz\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.341778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.336711    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqmhx\" (UniqueName: \"kubernetes.io/projected/c7179a4d-bc50-49c9-b605-27be4224c533-kube-api-access-vqmhx\") pod \"maas-api-revoke-sub-04d0ccf4-flksz\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.434727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.434636    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:28.787947 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:28.787921    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"]
Oct 30 13:18:29.662367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:29.661782    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz" event={"ID":"c7179a4d-bc50-49c9-b605-27be4224c533","Type":"ContainerStarted","Data":"abd5e062c392d028498e596f06b2088d947b402c619fe9f22ddde39d1f224a56"}
Oct 30 13:18:29.662367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:29.661885    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz" event={"ID":"c7179a4d-bc50-49c9-b605-27be4224c533","Type":"ContainerStarted","Data":"384e97a266d93dcd6a2c519be4104eab840eb4da928b0189ec16379590cc2a55"}
Oct 30 13:18:29.702200 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:29.702155    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz" podStartSLOduration=1.7021437590000001 podStartE2EDuration="1.702143759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:29.701900097 +0000 UTC m=+1246.999117641" watchObservedRunningTime="2026-10-30 13:18:29.702143759 +0000 UTC m=+1246.999361300"
Oct 30 13:18:29.896943 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:29.896904    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:30.671072 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:30.671041    5781 generic.go:356] "Generic (PLEG): container finished" podID="c7179a4d-bc50-49c9-b605-27be4224c533" containerID="abd5e062c392d028498e596f06b2088d947b402c619fe9f22ddde39d1f224a56" exitCode=0
Oct 30 13:18:30.671519 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:30.671126    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz" event={"ID":"c7179a4d-bc50-49c9-b605-27be4224c533","Type":"ContainerDied","Data":"abd5e062c392d028498e596f06b2088d947b402c619fe9f22ddde39d1f224a56"}
Oct 30 13:18:31.257012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.256976    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:31.499610 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.499576    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:31.844623 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.844601    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:31.930075 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.930044    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-xks-ca\") pod \"c7179a4d-bc50-49c9-b605-27be4224c533\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") "
Oct 30 13:18:31.930281 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.930136    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-openshift-ca\") pod \"c7179a4d-bc50-49c9-b605-27be4224c533\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") "
Oct 30 13:18:31.930281 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.930223    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7179a4d-bc50-49c9-b605-27be4224c533-kube-api-access-vqmhx\" (UniqueName: \"kubernetes.io/projected/c7179a4d-bc50-49c9-b605-27be4224c533-kube-api-access-vqmhx\") pod \"c7179a4d-bc50-49c9-b605-27be4224c533\" (UID: \"c7179a4d-bc50-49c9-b605-27be4224c533\") "
Oct 30 13:18:31.930587 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.930559    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-openshift-ca" pod "c7179a4d-bc50-49c9-b605-27be4224c533" (UID: "c7179a4d-bc50-49c9-b605-27be4224c533"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:31.931035 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.930992    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:31.932522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.932482    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7179a4d-bc50-49c9-b605-27be4224c533-kube-api-access-vqmhx" pod "c7179a4d-bc50-49c9-b605-27be4224c533" (UID: "c7179a4d-bc50-49c9-b605-27be4224c533"). InnerVolumeSpecName "kube-api-access-vqmhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:31.932755 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:31.932729    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-xks-ca" pod "c7179a4d-bc50-49c9-b605-27be4224c533" (UID: "c7179a4d-bc50-49c9-b605-27be4224c533"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:31.997172 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:18:31.997130    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:18:32.032018 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:32.031990    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqmhx\" (UniqueName: \"kubernetes.io/projected/c7179a4d-bc50-49c9-b605-27be4224c533-kube-api-access-vqmhx\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:32.032018 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:32.032015    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c7179a4d-bc50-49c9-b605-27be4224c533-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:32.688027 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:32.687991    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"
Oct 30 13:18:32.688205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:32.688035    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz" event={"ID":"c7179a4d-bc50-49c9-b605-27be4224c533","Type":"ContainerDied","Data":"384e97a266d93dcd6a2c519be4104eab840eb4da928b0189ec16379590cc2a55"}
Oct 30 13:18:32.688205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:32.688090    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="384e97a266d93dcd6a2c519be4104eab840eb4da928b0189ec16379590cc2a55"
Oct 30 13:18:33.125941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:33.125844    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:34.575447 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:34.575414    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:36.299568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:36.298779    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:37.050263 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.050229    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"]
Oct 30 13:18:37.050927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.050912    5781 cpu_manager.go:414] "containerMap: removing container" podUID="c7179a4d-bc50-49c9-b605-27be4224c533" containerName="revoke-keys"
Oct 30 13:18:37.050997 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.050936    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7179a4d-bc50-49c9-b605-27be4224c533" containerName="revoke-keys"
Oct 30 13:18:37.051064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.051055    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7179a4d-bc50-49c9-b605-27be4224c533" containerName="revoke-keys"
Oct 30 13:18:37.055582 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.055557    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.098605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.098570    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"]
Oct 30 13:18:37.101831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.101792    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.102649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.101866    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96af49f4-900b-45ae-88a3-8126f203bf77-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.102649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.101948    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.102649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.102003    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-965vb\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-kube-api-access-965vb\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.102649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.102030    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.102649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.102059    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.102649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.102112    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.102649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.102152    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.102649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.102254    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96af49f4-900b-45ae-88a3-8126f203bf77-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.104043 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.104022    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.104278 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.104255    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"]
Oct 30 13:18:37.137931 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.137893    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"]
Oct 30 13:18:37.204246 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204213    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.204431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204256    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.204431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204279    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.204431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204412    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.204617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204450    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.204617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204542    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljzjn\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-kube-api-access-ljzjn\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.204617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204589    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96af49f4-900b-45ae-88a3-8126f203bf77-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.204724 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204703    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.204781 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204765    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.204839 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204798    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-965vb\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-kube-api-access-965vb\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.204839 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204828    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.204958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204874    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.204958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204898    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.204958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204930    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.205106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.204975    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.205106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.205005    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.205106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.205077    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.205257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.205116    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96af49f4-900b-45ae-88a3-8126f203bf77-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.205257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.205122    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.205257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.205139    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.205257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.205219    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.205471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.205357    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96af49f4-900b-45ae-88a3-8126f203bf77-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.205554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.205538    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.207128 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.207108    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.207394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.207377    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96af49f4-900b-45ae-88a3-8126f203bf77-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.229342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.229308    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.229540 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.229516    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-965vb\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-kube-api-access-965vb\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.306495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.306400    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.306495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.306438    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.306562    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.306631    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljzjn\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-kube-api-access-ljzjn\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.306720    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.306752    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.306829    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-certs\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.306848    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.306977    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.307020    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.307164    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.307209    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-data\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.307367    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istiod-ca-cert\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.307454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.307412    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-credential-socket\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.309062 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.309046    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-envoy\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.309223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.309207    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-podinfo\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.330057 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.330023    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljzjn\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-kube-api-access-ljzjn\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.330489 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.330464    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-token\") pod \"e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") " pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.371116 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.371076    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:37.423137 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.423103    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:37.704123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.704092    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"]
Oct 30 13:18:37.705943 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:18:37.705919    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96af49f4_900b_45ae_88a3_8126f203bf77.slice/crio-7f87cc8859f388450b0de4e4bc05986274969c113cd45e8e4fc2d13069ebce2c WatchSource:0}: Error finding container 7f87cc8859f388450b0de4e4bc05986274969c113cd45e8e4fc2d13069ebce2c: Status 404 returned error can't find the container with id 7f87cc8859f388450b0de4e4bc05986274969c113cd45e8e4fc2d13069ebce2c
Oct 30 13:18:37.708567 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.708532    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:18:37.708687 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.708617    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:18:37.708687 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.708663    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:18:37.744028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.743985    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" event={"ID":"96af49f4-900b-45ae-88a3-8126f203bf77","Type":"ContainerStarted","Data":"7f87cc8859f388450b0de4e4bc05986274969c113cd45e8e4fc2d13069ebce2c"}
Oct 30 13:18:37.766895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.766870    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"]
Oct 30 13:18:37.769440 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:18:37.769405    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod356e2333_8a7f_40ca_b102_0f4812c5ceb3.slice/crio-324d23d7069cd28c51a2e6ef741079670e0e37025848c8c4498c57645589ff77 WatchSource:0}: Error finding container 324d23d7069cd28c51a2e6ef741079670e0e37025848c8c4498c57645589ff77: Status 404 returned error can't find the container with id 324d23d7069cd28c51a2e6ef741079670e0e37025848c8c4498c57645589ff77
Oct 30 13:18:37.772025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.771987    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:18:37.772143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.772071    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:18:37.772143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.772108    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:18:37.803211 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:37.803170    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:38.659758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:38.659719    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:38.771986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:38.771625    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" event={"ID":"356e2333-8a7f-40ca-b102-0f4812c5ceb3","Type":"ContainerStarted","Data":"43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f"}
Oct 30 13:18:38.772702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:38.772326    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" event={"ID":"356e2333-8a7f-40ca-b102-0f4812c5ceb3","Type":"ContainerStarted","Data":"324d23d7069cd28c51a2e6ef741079670e0e37025848c8c4498c57645589ff77"}
Oct 30 13:18:38.773237 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:38.773218    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" event={"ID":"96af49f4-900b-45ae-88a3-8126f203bf77","Type":"ContainerStarted","Data":"bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111"}
Oct 30 13:18:38.828679 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:38.828600    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podStartSLOduration=1.828579671 podStartE2EDuration="1.828579671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:38.827375181 +0000 UTC m=+1256.124592728" watchObservedRunningTime="2026-10-30 13:18:38.828579671 +0000 UTC m=+1256.125797216"
Oct 30 13:18:38.897118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:38.897069    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podStartSLOduration=1.897055239 podStartE2EDuration="1.897055239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:38.892592384 +0000 UTC m=+1256.189809927" watchObservedRunningTime="2026-10-30 13:18:38.897055239 +0000 UTC m=+1256.194272836"
Oct 30 13:18:39.377989 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:39.377943    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:39.379765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:39.379737    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 13:18:39.379904 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:39.379795    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 13:18:39.442025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:39.441981    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:39.442202 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:39.442116    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:18:39.442202 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:39.442162    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 13:18:39.632833 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:39.632741    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:40.377955 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:40.377906    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 13:18:40.378599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:40.378566    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 13:18:40.427647 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:40.427612    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:18:40.427943 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:40.427888    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 13:18:41.223313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.217376    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:41.401320 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.399210    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 13:18:41.444324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.433080    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:41.444324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.433915    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 13:18:41.484522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.484404    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:41.499318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.490750    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:18:41.499655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.499615    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 13:18:41.517334 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.502900    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:41.652313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.641210    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:41.652313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:41.641591    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 13:18:42.170928 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.170873    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.223454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.223402    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.224064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.224031    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.224879 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.224859    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:18:42.374405 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.374366    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 13:18:42.378404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.375373    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 13:18:42.486498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.454688    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 13:18:42.486498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.455061    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": context deadline exceeded"
Oct 30 13:18:42.486498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.455488    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:18:42.486498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.457334    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:18:42.486498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.457369    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 13:18:42.525145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.524966    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.592321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.530405    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:18:42.592321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.531323    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.592321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.584410    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:18:42.718383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.644380    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:42.718383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.644795    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:18:42.718383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.645104    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:18:42.718383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.645160    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:18:42.718383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:42.646806    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.269324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.120413    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:43.269324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.181375    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:18:43.269324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.182526    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:18:43.388313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.362705    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:18:43.388313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.367372    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 13:18:43.678459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.596690    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 13:18:43.678459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.596754    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 13:18:43.678459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.635410    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:18:43.678459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.635514    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 13:18:43.756312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.755384    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.756312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.755483    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 13:18:43.756312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.755538    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:18:43.756312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.755580    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 13:18:43.860311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.859782    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 13:18:43.860311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.859907    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:43.935958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:43.911602    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:18:44.196952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.196847    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:44.369441 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.369395    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:18:44.406316 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.383724    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:18:44.411934 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.411899    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:18:44.561627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.543062    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 13:18:44.561627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.543120    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 13:18:44.561627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.553154    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:18:44.561627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.553209    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 13:18:44.787204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.787132    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:18:44.794809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.794783    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:18:44.796982 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.796952    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:18:44.797779 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.797755    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:18:44.808280 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.808197    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:18:44.824368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:44.812590    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:18:45.002412 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:45.002008    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:18:45.376582 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:45.376547    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 13:18:45.376793 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:45.376613    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 13:18:45.441236 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:45.441196    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:18:45.441545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:45.441513    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 13:18:45.471001 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:45.470965    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:46.070500 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:46.070467    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:46.372242 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:46.372138    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 13:18:46.372242 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:46.372196    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 13:18:46.425677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:46.425640    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 13:18:46.425952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:46.425922    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 13:18:47.213136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.213088    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:47.372142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.372096    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused" start-of-body=
Oct 30 13:18:47.372343 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.372167    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": dial tcp 10.134.0.95:15021: connect: connection refused"
Oct 30 13:18:47.734169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.734128    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"]
Oct 30 13:18:47.741356 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.741326    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:47.799247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.791911    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:47.799247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.792041    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:47.810312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.810257    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"]
Oct 30 13:18:47.852172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.852119    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4hdn\" (UniqueName: \"kubernetes.io/projected/9f824fe6-a082-49b5-8168-9d97134c3c4c-kube-api-access-j4hdn\") pod \"maas-api-revoke-sub-9147fa74-tlbxq\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:47.852716 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.852682    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9147fa74-tlbxq\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:47.853009 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.852915    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9147fa74-tlbxq\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:47.955587 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.955545    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9147fa74-tlbxq\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:47.955761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.955619    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9147fa74-tlbxq\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:47.955888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.955868    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4hdn\" (UniqueName: \"kubernetes.io/projected/9f824fe6-a082-49b5-8168-9d97134c3c4c-kube-api-access-j4hdn\") pod \"maas-api-revoke-sub-9147fa74-tlbxq\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:47.956526 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.956496    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9147fa74-tlbxq\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:47.957784 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.957755    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:47.959350 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.959325    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:47.959721 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.959698    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9147fa74-tlbxq\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:47.960204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.960180    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:18:47.985875 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:47.980668    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4hdn\" (UniqueName: \"kubernetes.io/projected/9f824fe6-a082-49b5-8168-9d97134c3c4c-kube-api-access-j4hdn\") pod \"maas-api-revoke-sub-9147fa74-tlbxq\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:48.133340 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:48.133282    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:48.375926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:48.375842    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:48.376335 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:48.376315    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:48.376941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:48.376920    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:18:48.475234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:48.475206    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"]
Oct 30 13:18:48.478139 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:18:48.478112    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f824fe6_a082_49b5_8168_9d97134c3c4c.slice/crio-6090b1b24ba8fbc13b321309a083a854a655fda68c4152f09602ce3197dfb8b7 WatchSource:0}: Error finding container 6090b1b24ba8fbc13b321309a083a854a655fda68c4152f09602ce3197dfb8b7: Status 404 returned error can't find the container with id 6090b1b24ba8fbc13b321309a083a854a655fda68c4152f09602ce3197dfb8b7
Oct 30 13:18:48.843178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:48.843149    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq" event={"ID":"9f824fe6-a082-49b5-8168-9d97134c3c4c","Type":"ContainerStarted","Data":"29f4a742694ea3ff226762c4b1a29961c029afdb493c02848f7d4bfd1d6cffd0"}
Oct 30 13:18:48.843310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:48.843192    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq" event={"ID":"9f824fe6-a082-49b5-8168-9d97134c3c4c","Type":"ContainerStarted","Data":"6090b1b24ba8fbc13b321309a083a854a655fda68c4152f09602ce3197dfb8b7"}
Oct 30 13:18:48.890342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:48.890280    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq" podStartSLOduration=1.890264102 podStartE2EDuration="1.890264102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:48.888478595 +0000 UTC m=+1266.185696136" watchObservedRunningTime="2026-10-30 13:18:48.890264102 +0000 UTC m=+1266.187481644"
Oct 30 13:18:48.991407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:48.991237    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:49.851883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:49.851847    5781 generic.go:356] "Generic (PLEG): container finished" podID="9f824fe6-a082-49b5-8168-9d97134c3c4c" containerID="29f4a742694ea3ff226762c4b1a29961c029afdb493c02848f7d4bfd1d6cffd0" exitCode=0
Oct 30 13:18:49.852358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:49.851926    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq" event={"ID":"9f824fe6-a082-49b5-8168-9d97134c3c4c","Type":"ContainerDied","Data":"29f4a742694ea3ff226762c4b1a29961c029afdb493c02848f7d4bfd1d6cffd0"}
Oct 30 13:18:50.479458 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:50.479421    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:51.011895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.011863    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:51.109146 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.109058    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-openshift-ca\") pod \"9f824fe6-a082-49b5-8168-9d97134c3c4c\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") "
Oct 30 13:18:51.109358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.109189    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-xks-ca\") pod \"9f824fe6-a082-49b5-8168-9d97134c3c4c\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") "
Oct 30 13:18:51.109358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.109342    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f824fe6-a082-49b5-8168-9d97134c3c4c-kube-api-access-j4hdn\" (UniqueName: \"kubernetes.io/projected/9f824fe6-a082-49b5-8168-9d97134c3c4c-kube-api-access-j4hdn\") pod \"9f824fe6-a082-49b5-8168-9d97134c3c4c\" (UID: \"9f824fe6-a082-49b5-8168-9d97134c3c4c\") "
Oct 30 13:18:51.109485 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.109449    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-openshift-ca" pod "9f824fe6-a082-49b5-8168-9d97134c3c4c" (UID: "9f824fe6-a082-49b5-8168-9d97134c3c4c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:51.109915 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.109894    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:51.111301 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.111268    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-xks-ca" pod "9f824fe6-a082-49b5-8168-9d97134c3c4c" (UID: "9f824fe6-a082-49b5-8168-9d97134c3c4c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:51.111372 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.111315    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f824fe6-a082-49b5-8168-9d97134c3c4c-kube-api-access-j4hdn" pod "9f824fe6-a082-49b5-8168-9d97134c3c4c" (UID: "9f824fe6-a082-49b5-8168-9d97134c3c4c"). InnerVolumeSpecName "kube-api-access-j4hdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:51.211236 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.211200    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9f824fe6-a082-49b5-8168-9d97134c3c4c-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:51.211236 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.211230    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4hdn\" (UniqueName: \"kubernetes.io/projected/9f824fe6-a082-49b5-8168-9d97134c3c4c-kube-api-access-j4hdn\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:51.867620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.867546    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq" event={"ID":"9f824fe6-a082-49b5-8168-9d97134c3c4c","Type":"ContainerDied","Data":"6090b1b24ba8fbc13b321309a083a854a655fda68c4152f09602ce3197dfb8b7"}
Oct 30 13:18:51.867620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.867585    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"
Oct 30 13:18:51.867620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:51.867594    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6090b1b24ba8fbc13b321309a083a854a655fda68c4152f09602ce3197dfb8b7"
Oct 30 13:18:52.006760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.006713    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:52.876574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.876542    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"]
Oct 30 13:18:52.877746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.877718    5781 cpu_manager.go:414] "containerMap: removing container" podUID="9f824fe6-a082-49b5-8168-9d97134c3c4c" containerName="revoke-keys"
Oct 30 13:18:52.877746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.877742    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f824fe6-a082-49b5-8168-9d97134c3c4c" containerName="revoke-keys"
Oct 30 13:18:52.878056 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.878039    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f824fe6-a082-49b5-8168-9d97134c3c4c" containerName="revoke-keys"
Oct 30 13:18:52.882727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.882706    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:52.901958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.901914    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"]
Oct 30 13:18:52.908663 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.908635    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:18:52.908866 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.908669    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:18:52.939257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.939222    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7r5b\" (UniqueName: \"kubernetes.io/projected/ac0313fc-6f80-42fe-a4e7-7b339f88a597-kube-api-access-p7r5b\") pod \"maas-api-revoke-sub-4d2e932a-jcjvf\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:52.939495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.939475    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4d2e932a-jcjvf\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:52.939761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:52.939736    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4d2e932a-jcjvf\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:53.040902 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.040864    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4d2e932a-jcjvf\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:53.041080 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.040943    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7r5b\" (UniqueName: \"kubernetes.io/projected/ac0313fc-6f80-42fe-a4e7-7b339f88a597-kube-api-access-p7r5b\") pod \"maas-api-revoke-sub-4d2e932a-jcjvf\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:53.041080 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.041031    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4d2e932a-jcjvf\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:53.041687 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.041664    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4d2e932a-jcjvf\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:53.043399 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.043369    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4d2e932a-jcjvf\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:53.064128 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.064093    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7r5b\" (UniqueName: \"kubernetes.io/projected/ac0313fc-6f80-42fe-a4e7-7b339f88a597-kube-api-access-p7r5b\") pod \"maas-api-revoke-sub-4d2e932a-jcjvf\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:53.197967 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.197931    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:53.432389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.432359    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:53.512991 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.512961    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"]
Oct 30 13:18:53.514686 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:18:53.514653    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac0313fc_6f80_42fe_a4e7_7b339f88a597.slice/crio-5201da7dc40a86f0299c13fbb7d1b8a544b22ec3d78e3b0f68315c6617eea08b WatchSource:0}: Error finding container 5201da7dc40a86f0299c13fbb7d1b8a544b22ec3d78e3b0f68315c6617eea08b: Status 404 returned error can't find the container with id 5201da7dc40a86f0299c13fbb7d1b8a544b22ec3d78e3b0f68315c6617eea08b
Oct 30 13:18:53.896751 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.896717    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf" event={"ID":"ac0313fc-6f80-42fe-a4e7-7b339f88a597","Type":"ContainerStarted","Data":"a6cba356a9865f2241c0798f6ca09ad83be3185bd0d2501be180ba83e5d84ed9"}
Oct 30 13:18:53.897172 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.896805    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf" event={"ID":"ac0313fc-6f80-42fe-a4e7-7b339f88a597","Type":"ContainerStarted","Data":"5201da7dc40a86f0299c13fbb7d1b8a544b22ec3d78e3b0f68315c6617eea08b"}
Oct 30 13:18:53.937989 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:53.937910    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf" podStartSLOduration=1.937897255 podStartE2EDuration="1.937897255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:18:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:18:53.936462509 +0000 UTC m=+1271.233680050" watchObservedRunningTime="2026-10-30 13:18:53.937897255 +0000 UTC m=+1271.235114797"
Oct 30 13:18:54.844479 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:54.844442    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:54.906966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:54.906932    5781 generic.go:356] "Generic (PLEG): container finished" podID="ac0313fc-6f80-42fe-a4e7-7b339f88a597" containerID="a6cba356a9865f2241c0798f6ca09ad83be3185bd0d2501be180ba83e5d84ed9" exitCode=0
Oct 30 13:18:54.907426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:54.907016    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf" event={"ID":"ac0313fc-6f80-42fe-a4e7-7b339f88a597","Type":"ContainerDied","Data":"a6cba356a9865f2241c0798f6ca09ad83be3185bd0d2501be180ba83e5d84ed9"}
Oct 30 13:18:56.064965 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.064937    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:56.127331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.127272    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:56.223310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.222847    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-openshift-ca\") pod \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") "
Oct 30 13:18:56.223310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.222987    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac0313fc-6f80-42fe-a4e7-7b339f88a597-kube-api-access-p7r5b\" (UniqueName: \"kubernetes.io/projected/ac0313fc-6f80-42fe-a4e7-7b339f88a597-kube-api-access-p7r5b\") pod \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") "
Oct 30 13:18:56.223310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.223143    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-xks-ca\") pod \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\" (UID: \"ac0313fc-6f80-42fe-a4e7-7b339f88a597\") "
Oct 30 13:18:56.232995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.232961    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-openshift-ca" pod "ac0313fc-6f80-42fe-a4e7-7b339f88a597" (UID: "ac0313fc-6f80-42fe-a4e7-7b339f88a597"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:56.268307 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.253407    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-xks-ca" pod "ac0313fc-6f80-42fe-a4e7-7b339f88a597" (UID: "ac0313fc-6f80-42fe-a4e7-7b339f88a597"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:56.268307 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.266793    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac0313fc-6f80-42fe-a4e7-7b339f88a597-kube-api-access-p7r5b" pod "ac0313fc-6f80-42fe-a4e7-7b339f88a597" (UID: "ac0313fc-6f80-42fe-a4e7-7b339f88a597"). InnerVolumeSpecName "kube-api-access-p7r5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:56.288150 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.288115    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"]
Oct 30 13:18:56.289230 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.289209    5781 cpu_manager.go:414] "containerMap: removing container" podUID="ac0313fc-6f80-42fe-a4e7-7b339f88a597" containerName="revoke-keys"
Oct 30 13:18:56.289324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.289233    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac0313fc-6f80-42fe-a4e7-7b339f88a597" containerName="revoke-keys"
Oct 30 13:18:56.289417 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.289404    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac0313fc-6f80-42fe-a4e7-7b339f88a597" containerName="revoke-keys"
Oct 30 13:18:56.293634 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.293610    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.314337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.314307    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"]
Oct 30 13:18:56.325216 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.325188    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:56.325419 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.325401    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7r5b\" (UniqueName: \"kubernetes.io/projected/ac0313fc-6f80-42fe-a4e7-7b339f88a597-kube-api-access-p7r5b\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:56.325558 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.325545    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ac0313fc-6f80-42fe-a4e7-7b339f88a597-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:56.426837 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.426798    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.427012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.426843    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.427062 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.427025    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwss8\" (UniqueName: \"kubernetes.io/projected/c3d96339-09a6-4e5e-aca4-af1b4b454fff-kube-api-access-lwss8\") pod \"maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.528734 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.528645    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwss8\" (UniqueName: \"kubernetes.io/projected/c3d96339-09a6-4e5e-aca4-af1b4b454fff-kube-api-access-lwss8\") pod \"maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.528884 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.528866    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.528923 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.528905    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.529624 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.529566    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.531305 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.531261    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.553029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.553009    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwss8\" (UniqueName: \"kubernetes.io/projected/c3d96339-09a6-4e5e-aca4-af1b4b454fff-kube-api-access-lwss8\") pod \"maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.612749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.612719    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:56.938974 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.938948    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"
Oct 30 13:18:56.939174 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.938944    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf" event={"ID":"ac0313fc-6f80-42fe-a4e7-7b339f88a597","Type":"ContainerDied","Data":"5201da7dc40a86f0299c13fbb7d1b8a544b22ec3d78e3b0f68315c6617eea08b"}
Oct 30 13:18:56.939174 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.939082    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5201da7dc40a86f0299c13fbb7d1b8a544b22ec3d78e3b0f68315c6617eea08b"
Oct 30 13:18:56.972875 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:56.972852    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"]
Oct 30 13:18:56.973998 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:18:56.973967    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d96339_09a6_4e5e_aca4_af1b4b454fff.slice/crio-b7a57c58a7588115418a27a8757d6adb873e856f18323e4f677a72b96af413c5 WatchSource:0}: Error finding container b7a57c58a7588115418a27a8757d6adb873e856f18323e4f677a72b96af413c5: Status 404 returned error can't find the container with id b7a57c58a7588115418a27a8757d6adb873e856f18323e4f677a72b96af413c5
Oct 30 13:18:57.087431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:57.087391    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:57.948013 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:57.947975    5781 generic.go:356] "Generic (PLEG): container finished" podID="c3d96339-09a6-4e5e-aca4-af1b4b454fff" containerID="14abef82f64b11732e68669de146fdbe9e6b9894c74d9d05e94ea88d569e9461" exitCode=0
Oct 30 13:18:57.948192 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:57.948042    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz" event={"ID":"c3d96339-09a6-4e5e-aca4-af1b4b454fff","Type":"ContainerDied","Data":"14abef82f64b11732e68669de146fdbe9e6b9894c74d9d05e94ea88d569e9461"}
Oct 30 13:18:57.948192 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:57.948088    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz" event={"ID":"c3d96339-09a6-4e5e-aca4-af1b4b454fff","Type":"ContainerStarted","Data":"b7a57c58a7588115418a27a8757d6adb873e856f18323e4f677a72b96af413c5"}
Oct 30 13:18:58.751099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:58.750965    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:18:59.118146 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.118126    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:59.269427 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.269356    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-openshift-ca\") pod \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") "
Oct 30 13:18:59.269427 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.269417    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-xks-ca\") pod \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") "
Oct 30 13:18:59.269581 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.269554    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3d96339-09a6-4e5e-aca4-af1b4b454fff-kube-api-access-lwss8\" (UniqueName: \"kubernetes.io/projected/c3d96339-09a6-4e5e-aca4-af1b4b454fff-kube-api-access-lwss8\") pod \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\" (UID: \"c3d96339-09a6-4e5e-aca4-af1b4b454fff\") "
Oct 30 13:18:59.269785 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.269760    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-openshift-ca" pod "c3d96339-09a6-4e5e-aca4-af1b4b454fff" (UID: "c3d96339-09a6-4e5e-aca4-af1b4b454fff"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:18:59.269992 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.269967    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:59.271541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.271514    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-xks-ca" pod "c3d96339-09a6-4e5e-aca4-af1b4b454fff" (UID: "c3d96339-09a6-4e5e-aca4-af1b4b454fff"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:18:59.271651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.271632    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d96339-09a6-4e5e-aca4-af1b4b454fff-kube-api-access-lwss8" pod "c3d96339-09a6-4e5e-aca4-af1b4b454fff" (UID: "c3d96339-09a6-4e5e-aca4-af1b4b454fff"). InnerVolumeSpecName "kube-api-access-lwss8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:18:59.371196 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.371165    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c3d96339-09a6-4e5e-aca4-af1b4b454fff-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:59.371196 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.371191    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwss8\" (UniqueName: \"kubernetes.io/projected/c3d96339-09a6-4e5e-aca4-af1b4b454fff-kube-api-access-lwss8\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:18:59.963141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.963112    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"
Oct 30 13:18:59.963540 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.963146    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz" event={"ID":"c3d96339-09a6-4e5e-aca4-af1b4b454fff","Type":"ContainerDied","Data":"b7a57c58a7588115418a27a8757d6adb873e856f18323e4f677a72b96af413c5"}
Oct 30 13:18:59.963540 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:18:59.963200    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b7a57c58a7588115418a27a8757d6adb873e856f18323e4f677a72b96af413c5"
Oct 30 13:19:00.292520 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:00.292423    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:01.088363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:01.081225    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"]
Oct 30 13:19:01.122313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:01.121904    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"]
Oct 30 13:19:01.295312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:01.287658    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:02.350320 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:02.349581    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:19:02.399884 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.399829    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:02.411433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.411360    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:02.435133 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.435078    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:02.639365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.638384    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:02.760434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.760393    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:19:02.762131 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.762071    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:02.815313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.812875    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:02.815599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.815568    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": context deadline exceeded"
Oct 30 13:19:02.907485 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.906426    5781 patch_prober.go:29] interesting pod/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:02.907751 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.907709    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.90:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:02.994444 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:02.994396    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.465441 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.465383    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.486381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.486349    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:19:03.547328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.508200    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.547328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.542397    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:19:03.737370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.726403    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.762689 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.762646    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 13:19:03.795502 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.795458    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:19:03.797219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.797193    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:19:03.885626 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.885594    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:19:03.886748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.845396    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 13:19:03.887442 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.845909    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:03.888135 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.888117    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:19:03.888304 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:03.888275    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:19:04.004169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.004079    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.006198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.006176    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:19:04.226313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.060637    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:04.281816 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.281714    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.498342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.435733    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:04.498342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.436348    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.516309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.499267    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.559590 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.559493    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:04.715107 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.715058    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 13:19:04.829466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.794761    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:05.081393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.849768    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:19:05.081393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.922439    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:05.081393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.922509    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 13:19:05.081393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:04.950849    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:05.160313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.159401    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:19:05.220667 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.220624    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:05.363546 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.363465    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:19:05.391865 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.391835    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:19:05.448641 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.448608    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:05.449667 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.449643    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:19:05.449866 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.449856    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:19:05.479703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.479671    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:19:05.479888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.479758    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:19:05.485416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:05.485338    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:19:06.248140 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.248106    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"]
Oct 30 13:19:06.248768 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.248754    5781 cpu_manager.go:414] "containerMap: removing container" podUID="c3d96339-09a6-4e5e-aca4-af1b4b454fff" containerName="revoke-keys"
Oct 30 13:19:06.248816 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.248769    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3d96339-09a6-4e5e-aca4-af1b4b454fff" containerName="revoke-keys"
Oct 30 13:19:06.248918 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.248909    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3d96339-09a6-4e5e-aca4-af1b4b454fff" containerName="revoke-keys"
Oct 30 13:19:06.252750 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.252734    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.271229 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.271199    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq5j6\" (UniqueName: \"kubernetes.io/projected/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-kube-api-access-qq5j6\") pod \"maas-api-revoke-sub-f0ef9c8e-lnr9x\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.271431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.271413    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f0ef9c8e-lnr9x\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.271505 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.271482    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f0ef9c8e-lnr9x\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.272655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.272634    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:06.284394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.284373    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:06.289642 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.289618    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"]
Oct 30 13:19:06.372873 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.372840    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f0ef9c8e-lnr9x\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.373041 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.372922    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f0ef9c8e-lnr9x\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.373112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.373055    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq5j6\" (UniqueName: \"kubernetes.io/projected/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-kube-api-access-qq5j6\") pod \"maas-api-revoke-sub-f0ef9c8e-lnr9x\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.373612 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.373586    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f0ef9c8e-lnr9x\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.375262 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.375239    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f0ef9c8e-lnr9x\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.408355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.408329    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq5j6\" (UniqueName: \"kubernetes.io/projected/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-kube-api-access-qq5j6\") pod \"maas-api-revoke-sub-f0ef9c8e-lnr9x\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.566263 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.566162    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:06.688046 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.688002    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"]
Oct 30 13:19:06.899089 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:19:06.899051    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab4d4ab3_8a37_4774_a85e_4108eb8354eb.slice/crio-f96be0fbeef9a04eea95387fa8199ac1f93a2ab299dfced23c564b56693445f0 WatchSource:0}: Error finding container f96be0fbeef9a04eea95387fa8199ac1f93a2ab299dfced23c564b56693445f0: Status 404 returned error can't find the container with id f96be0fbeef9a04eea95387fa8199ac1f93a2ab299dfced23c564b56693445f0
Oct 30 13:19:06.916570 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:06.914527    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"]
Oct 30 13:19:07.019606 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:07.019574    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x" event={"ID":"ab4d4ab3-8a37-4774-a85e-4108eb8354eb","Type":"ContainerStarted","Data":"f96be0fbeef9a04eea95387fa8199ac1f93a2ab299dfced23c564b56693445f0"}
Oct 30 13:19:07.076661 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:07.076601    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:07.361239 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:07.361206    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:07.877889 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:07.877860    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"]
Oct 30 13:19:07.882358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:07.882325    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:07.904876 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:07.904849    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zctvg\" (UniqueName: \"kubernetes.io/projected/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-kube-api-access-zctvg\") pod \"maas-api-revoke-sub-61ad8f8b-mqszs\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:07.905124 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:07.905096    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-xks-ca\") pod \"maas-api-revoke-sub-61ad8f8b-mqszs\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:07.905342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:07.905315    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-61ad8f8b-mqszs\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:07.906912 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:07.906890    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"]
Oct 30 13:19:08.006907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.006870    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-xks-ca\") pod \"maas-api-revoke-sub-61ad8f8b-mqszs\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:08.007086 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.007009    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-61ad8f8b-mqszs\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:08.007130 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.007110    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zctvg\" (UniqueName: \"kubernetes.io/projected/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-kube-api-access-zctvg\") pod \"maas-api-revoke-sub-61ad8f8b-mqszs\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:08.007732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.007701    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-61ad8f8b-mqszs\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:08.009266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.009249    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-xks-ca\") pod \"maas-api-revoke-sub-61ad8f8b-mqszs\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:08.028191 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.028161    5781 generic.go:356] "Generic (PLEG): container finished" podID="ab4d4ab3-8a37-4774-a85e-4108eb8354eb" containerID="a82e83669973de34cc7d76a7be25b0453fda74fc763f36b4aeec2367e3e48623" exitCode=0
Oct 30 13:19:08.028371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.028233    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x" event={"ID":"ab4d4ab3-8a37-4774-a85e-4108eb8354eb","Type":"ContainerDied","Data":"a82e83669973de34cc7d76a7be25b0453fda74fc763f36b4aeec2367e3e48623"}
Oct 30 13:19:08.030745 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.030726    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zctvg\" (UniqueName: \"kubernetes.io/projected/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-kube-api-access-zctvg\") pod \"maas-api-revoke-sub-61ad8f8b-mqszs\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:08.203805 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.203767    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:08.509322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.509278    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"]
Oct 30 13:19:08.511147 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:19:08.511121    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059dcfe8_67c7_4d33_9b6b_f49cc4dd9b43.slice/crio-c22a8b578ed562cebaf14de45e8892dd46fa6f2fa2725567ecaf729a8900ef9d WatchSource:0}: Error finding container c22a8b578ed562cebaf14de45e8892dd46fa6f2fa2725567ecaf729a8900ef9d: Status 404 returned error can't find the container with id c22a8b578ed562cebaf14de45e8892dd46fa6f2fa2725567ecaf729a8900ef9d
Oct 30 13:19:08.901797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:08.901761    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:09.047053 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.047015    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs" event={"ID":"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43","Type":"ContainerStarted","Data":"18ad39283c5ee803a22e6790831da42f0459a01801f9d89e8246a04b45a49672"}
Oct 30 13:19:09.047224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.047104    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs" event={"ID":"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43","Type":"ContainerStarted","Data":"c22a8b578ed562cebaf14de45e8892dd46fa6f2fa2725567ecaf729a8900ef9d"}
Oct 30 13:19:09.088711 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.088604    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs" podStartSLOduration=2.088589933 podStartE2EDuration="2.088589933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:09.086487159 +0000 UTC m=+1286.383704713" watchObservedRunningTime="2026-10-30 13:19:09.088589933 +0000 UTC m=+1286.385807474"
Oct 30 13:19:09.201546 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.201522    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:09.325898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.325866    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-kube-api-access-qq5j6\" (UniqueName: \"kubernetes.io/projected/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-kube-api-access-qq5j6\") pod \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") "
Oct 30 13:19:09.326066 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.325946    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-openshift-ca\") pod \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") "
Oct 30 13:19:09.326066 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.326037    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-xks-ca\") pod \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\" (UID: \"ab4d4ab3-8a37-4774-a85e-4108eb8354eb\") "
Oct 30 13:19:09.326401 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.326371    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-openshift-ca" pod "ab4d4ab3-8a37-4774-a85e-4108eb8354eb" (UID: "ab4d4ab3-8a37-4774-a85e-4108eb8354eb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:09.326853 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.326822    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:09.328131 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.328101    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-xks-ca" pod "ab4d4ab3-8a37-4774-a85e-4108eb8354eb" (UID: "ab4d4ab3-8a37-4774-a85e-4108eb8354eb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:09.328131 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.328111    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-kube-api-access-qq5j6" pod "ab4d4ab3-8a37-4774-a85e-4108eb8354eb" (UID: "ab4d4ab3-8a37-4774-a85e-4108eb8354eb"). InnerVolumeSpecName "kube-api-access-qq5j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:09.428122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.428085    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq5j6\" (UniqueName: \"kubernetes.io/projected/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-kube-api-access-qq5j6\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:09.428122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:09.428112    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ab4d4ab3-8a37-4774-a85e-4108eb8354eb-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:10.056881 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:10.056851    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"
Oct 30 13:19:10.057371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:10.056888    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x" event={"ID":"ab4d4ab3-8a37-4774-a85e-4108eb8354eb","Type":"ContainerDied","Data":"f96be0fbeef9a04eea95387fa8199ac1f93a2ab299dfced23c564b56693445f0"}
Oct 30 13:19:10.057371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:10.056935    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f96be0fbeef9a04eea95387fa8199ac1f93a2ab299dfced23c564b56693445f0"
Oct 30 13:19:10.058495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:10.058466    5781 generic.go:356] "Generic (PLEG): container finished" podID="059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43" containerID="18ad39283c5ee803a22e6790831da42f0459a01801f9d89e8246a04b45a49672" exitCode=0
Oct 30 13:19:10.058627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:10.058556    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs" event={"ID":"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43","Type":"ContainerDied","Data":"18ad39283c5ee803a22e6790831da42f0459a01801f9d89e8246a04b45a49672"}
Oct 30 13:19:10.375708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:10.375618    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:11.285176 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.285139    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:11.353975 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.353895    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-kube-api-access-zctvg\" (UniqueName: \"kubernetes.io/projected/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-kube-api-access-zctvg\") pod \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") "
Oct 30 13:19:11.354273 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.354250    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-xks-ca\") pod \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") "
Oct 30 13:19:11.354524 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.354505    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-openshift-ca\") pod \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\" (UID: \"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43\") "
Oct 30 13:19:11.355027 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.355002    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-openshift-ca" pod "059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43" (UID: "059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:11.355606 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.355583    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:11.357384 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.356781    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-kube-api-access-zctvg" pod "059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43" (UID: "059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43"). InnerVolumeSpecName "kube-api-access-zctvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:11.357384 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.357630    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-xks-ca" pod "059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43" (UID: "059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:11.457310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.457269    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zctvg\" (UniqueName: \"kubernetes.io/projected/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-kube-api-access-zctvg\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:11.457310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.457311    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:11.689063 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.688993    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx" podUID="4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" containerName="payload-processing" containerID="cri-o://09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1" gracePeriod=25
Oct 30 13:19:11.787784 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.787738    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"]
Oct 30 13:19:11.788766 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.788742    5781 cpu_manager.go:414] "containerMap: removing container" podUID="059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43" containerName="revoke-keys"
Oct 30 13:19:11.788766 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.788764    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43" containerName="revoke-keys"
Oct 30 13:19:11.788921 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.788815    5781 cpu_manager.go:414] "containerMap: removing container" podUID="ab4d4ab3-8a37-4774-a85e-4108eb8354eb" containerName="revoke-keys"
Oct 30 13:19:11.788921 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.788820    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab4d4ab3-8a37-4774-a85e-4108eb8354eb" containerName="revoke-keys"
Oct 30 13:19:11.788921 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.788916    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab4d4ab3-8a37-4774-a85e-4108eb8354eb" containerName="revoke-keys"
Oct 30 13:19:11.789010 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.788927    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43" containerName="revoke-keys"
Oct 30 13:19:11.792868 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.792852    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:11.817676 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.815500    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"]
Oct 30 13:19:11.862819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.862777    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9tlp\" (UniqueName: \"kubernetes.io/projected/b92d8ba7-d527-4258-b054-18a1f5d112e5-kube-api-access-k9tlp\") pod \"maas-api-revoke-sub-0f00f912-ptrn4\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:11.862986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.862923    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0f00f912-ptrn4\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:11.863064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.863035    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0f00f912-ptrn4\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:11.964391 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.964276    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9tlp\" (UniqueName: \"kubernetes.io/projected/b92d8ba7-d527-4258-b054-18a1f5d112e5-kube-api-access-k9tlp\") pod \"maas-api-revoke-sub-0f00f912-ptrn4\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:11.964553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.964457    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0f00f912-ptrn4\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:11.964553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.964523    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0f00f912-ptrn4\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:11.965308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.965266    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0f00f912-ptrn4\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:11.966802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.966784    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0f00f912-ptrn4\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:11.987703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:11.987661    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9tlp\" (UniqueName: \"kubernetes.io/projected/b92d8ba7-d527-4258-b054-18a1f5d112e5-kube-api-access-k9tlp\") pod \"maas-api-revoke-sub-0f00f912-ptrn4\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:12.075098 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:12.075069    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"
Oct 30 13:19:12.075261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:12.075100    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs" event={"ID":"059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43","Type":"ContainerDied","Data":"c22a8b578ed562cebaf14de45e8892dd46fa6f2fa2725567ecaf729a8900ef9d"}
Oct 30 13:19:12.075261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:12.075151    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c22a8b578ed562cebaf14de45e8892dd46fa6f2fa2725567ecaf729a8900ef9d"
Oct 30 13:19:12.111691 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:12.111664    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:12.230428 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:12.230273    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:12.420241 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:12.420218    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"]
Oct 30 13:19:12.421813 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:19:12.421784    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92d8ba7_d527_4258_b054_18a1f5d112e5.slice/crio-a321bd29f69c799e1cfbcae8cb266c069a6750d9fa09e3047c887a1bc0f9ca35 WatchSource:0}: Error finding container a321bd29f69c799e1cfbcae8cb266c069a6750d9fa09e3047c887a1bc0f9ca35: Status 404 returned error can't find the container with id a321bd29f69c799e1cfbcae8cb266c069a6750d9fa09e3047c887a1bc0f9ca35
Oct 30 13:19:13.083968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:13.083937    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4" event={"ID":"b92d8ba7-d527-4258-b054-18a1f5d112e5","Type":"ContainerStarted","Data":"c7876ff084237f8c2d070efaf92ff11ec8575fd7cc5183da4eb1925169b52784"}
Oct 30 13:19:13.084173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:13.083985    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4" event={"ID":"b92d8ba7-d527-4258-b054-18a1f5d112e5","Type":"ContainerStarted","Data":"a321bd29f69c799e1cfbcae8cb266c069a6750d9fa09e3047c887a1bc0f9ca35"}
Oct 30 13:19:13.830564 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:13.830524    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:14.093034 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:14.092954    5781 generic.go:356] "Generic (PLEG): container finished" podID="b92d8ba7-d527-4258-b054-18a1f5d112e5" containerID="c7876ff084237f8c2d070efaf92ff11ec8575fd7cc5183da4eb1925169b52784" exitCode=0
Oct 30 13:19:14.093174 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:14.093035    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4" event={"ID":"b92d8ba7-d527-4258-b054-18a1f5d112e5","Type":"ContainerDied","Data":"c7876ff084237f8c2d070efaf92ff11ec8575fd7cc5183da4eb1925169b52784"}
Oct 30 13:19:15.290382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.290359    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:15.394777 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.394686    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:15.417722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.417681    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b92d8ba7-d527-4258-b054-18a1f5d112e5-kube-api-access-k9tlp\" (UniqueName: \"kubernetes.io/projected/b92d8ba7-d527-4258-b054-18a1f5d112e5-kube-api-access-k9tlp\") pod \"b92d8ba7-d527-4258-b054-18a1f5d112e5\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") "
Oct 30 13:19:15.417888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.417823    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-openshift-ca\") pod \"b92d8ba7-d527-4258-b054-18a1f5d112e5\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") "
Oct 30 13:19:15.417940 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.417892    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-xks-ca\") pod \"b92d8ba7-d527-4258-b054-18a1f5d112e5\" (UID: \"b92d8ba7-d527-4258-b054-18a1f5d112e5\") "
Oct 30 13:19:15.418241 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.418211    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-openshift-ca" pod "b92d8ba7-d527-4258-b054-18a1f5d112e5" (UID: "b92d8ba7-d527-4258-b054-18a1f5d112e5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:15.418574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.418555    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:15.420215 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.420183    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b92d8ba7-d527-4258-b054-18a1f5d112e5-kube-api-access-k9tlp" pod "b92d8ba7-d527-4258-b054-18a1f5d112e5" (UID: "b92d8ba7-d527-4258-b054-18a1f5d112e5"). InnerVolumeSpecName "kube-api-access-k9tlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:15.420353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.420262    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-xks-ca" pod "b92d8ba7-d527-4258-b054-18a1f5d112e5" (UID: "b92d8ba7-d527-4258-b054-18a1f5d112e5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:15.520507 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.520476    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9tlp\" (UniqueName: \"kubernetes.io/projected/b92d8ba7-d527-4258-b054-18a1f5d112e5-kube-api-access-k9tlp\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:15.520507 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:15.520504    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b92d8ba7-d527-4258-b054-18a1f5d112e5-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:16.128600 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.128550    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4" event={"ID":"b92d8ba7-d527-4258-b054-18a1f5d112e5","Type":"ContainerDied","Data":"a321bd29f69c799e1cfbcae8cb266c069a6750d9fa09e3047c887a1bc0f9ca35"}
Oct 30 13:19:16.128806 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.128625    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a321bd29f69c799e1cfbcae8cb266c069a6750d9fa09e3047c887a1bc0f9ca35"
Oct 30 13:19:16.128806 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.128692    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"
Oct 30 13:19:16.765311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.764755    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"]
Oct 30 13:19:16.769309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.765853    5781 cpu_manager.go:414] "containerMap: removing container" podUID="b92d8ba7-d527-4258-b054-18a1f5d112e5" containerName="revoke-keys"
Oct 30 13:19:16.769309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.765876    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b92d8ba7-d527-4258-b054-18a1f5d112e5" containerName="revoke-keys"
Oct 30 13:19:16.769309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.766223    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="b92d8ba7-d527-4258-b054-18a1f5d112e5" containerName="revoke-keys"
Oct 30 13:19:16.776345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.771123    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:16.802993 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.802933    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"]
Oct 30 13:19:16.805487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.805468    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:16.805680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.805663    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:16.878127 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.878084    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b06036fa-46vqf\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:16.887905 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.887861    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7r22\" (UniqueName: \"kubernetes.io/projected/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-kube-api-access-c7r22\") pod \"maas-api-revoke-sub-b06036fa-46vqf\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:16.888064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.887989    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b06036fa-46vqf\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:16.991841 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.991783    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b06036fa-46vqf\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:16.992007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.991883    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7r22\" (UniqueName: \"kubernetes.io/projected/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-kube-api-access-c7r22\") pod \"maas-api-revoke-sub-b06036fa-46vqf\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:16.992086 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.992009    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b06036fa-46vqf\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:16.993454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:16.993427    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b06036fa-46vqf\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:17.069354 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:17.062803    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b06036fa-46vqf\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:17.487514 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:17.476349    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7r22\" (UniqueName: \"kubernetes.io/projected/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-kube-api-access-c7r22\") pod \"maas-api-revoke-sub-b06036fa-46vqf\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:17.727408 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:17.725196    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:18.117771 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.117392    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx" podUID="4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:18.206378 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.205685    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:18.654545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.586618    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:18.654545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.624487    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 13:19:18.654545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.627674    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:18.654545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.636754    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 13:19:18.671394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.655043    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:18.763573 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.761737    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": context deadline exceeded"
Oct 30 13:19:18.984854 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.892368    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.84:8000/ready\": context deadline exceeded"
Oct 30 13:19:18.984854 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.903373    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:19:18.984854 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.906891    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:18.984854 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:18.914188    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 13:19:18.986660 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:19:18.985992    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63254bd2_cb27_4bd6_a4c4_5562a9d6262c.slice/crio-2200569ad9f7277aafde80f89548f07d8752e6dc8506e3ddc0233b03ffd2c025 WatchSource:0}: Error finding container 2200569ad9f7277aafde80f89548f07d8752e6dc8506e3ddc0233b03ffd2c025: Status 404 returned error can't find the container with id 2200569ad9f7277aafde80f89548f07d8752e6dc8506e3ddc0233b03ffd2c025
Oct 30 13:19:19.009850 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:19.009815    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"]
Oct 30 13:19:20.005243 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.005212    5781 generic.go:356] "Generic (PLEG): container finished" podID="63254bd2-cb27-4bd6-a4c4-5562a9d6262c" containerID="66d4bb58e6b25420f891d3647ebe21c65c202183f1bac02cf78d6cadb8b19511" exitCode=0
Oct 30 13:19:20.005613 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.005323    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf" event={"ID":"63254bd2-cb27-4bd6-a4c4-5562a9d6262c","Type":"ContainerDied","Data":"66d4bb58e6b25420f891d3647ebe21c65c202183f1bac02cf78d6cadb8b19511"}
Oct 30 13:19:20.005613 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.005371    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf" event={"ID":"63254bd2-cb27-4bd6-a4c4-5562a9d6262c","Type":"ContainerStarted","Data":"2200569ad9f7277aafde80f89548f07d8752e6dc8506e3ddc0233b03ffd2c025"}
Oct 30 13:19:20.028127 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.028058    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main" containerID="cri-o://7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81" gracePeriod=42
Oct 30 13:19:20.310825 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.310791    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:19:20.327420 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.327389    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-home\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-home\") pod \"b724a211-f022-417f-aab0-2f78bab05fbd\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") "
Oct 30 13:19:20.327592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.327482    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b724a211-f022-417f-aab0-2f78bab05fbd-kube-api-access-5w595\" (UniqueName: \"kubernetes.io/projected/b724a211-f022-417f-aab0-2f78bab05fbd-kube-api-access-5w595\") pod \"b724a211-f022-417f-aab0-2f78bab05fbd\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") "
Oct 30 13:19:20.327686 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.327589    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-model-cache\") pod \"b724a211-f022-417f-aab0-2f78bab05fbd\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") "
Oct 30 13:19:20.327686 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.327656    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b724a211-f022-417f-aab0-2f78bab05fbd-tls-certs\" (UniqueName: \"kubernetes.io/secret/b724a211-f022-417f-aab0-2f78bab05fbd-tls-certs\") pod \"b724a211-f022-417f-aab0-2f78bab05fbd\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") "
Oct 30 13:19:20.327803 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.327676    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-home" pod "b724a211-f022-417f-aab0-2f78bab05fbd" (UID: "b724a211-f022-417f-aab0-2f78bab05fbd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:20.327803 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.327706    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-tmp-dir\") pod \"b724a211-f022-417f-aab0-2f78bab05fbd\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") "
Oct 30 13:19:20.327803 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.327736    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-dshm\") pod \"b724a211-f022-417f-aab0-2f78bab05fbd\" (UID: \"b724a211-f022-417f-aab0-2f78bab05fbd\") "
Oct 30 13:19:20.327946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.327862    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-model-cache" pod "b724a211-f022-417f-aab0-2f78bab05fbd" (UID: "b724a211-f022-417f-aab0-2f78bab05fbd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:20.328210 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.328184    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-tmp-dir" pod "b724a211-f022-417f-aab0-2f78bab05fbd" (UID: "b724a211-f022-417f-aab0-2f78bab05fbd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:20.328583 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.328563    5781 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-home\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.328696 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.328590    5781 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-model-cache\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.328696 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.328605    5781 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-tmp-dir\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.335160 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.335135    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b724a211-f022-417f-aab0-2f78bab05fbd-tls-certs" pod "b724a211-f022-417f-aab0-2f78bab05fbd" (UID: "b724a211-f022-417f-aab0-2f78bab05fbd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:20.335246 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.335217    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-dshm" pod "b724a211-f022-417f-aab0-2f78bab05fbd" (UID: "b724a211-f022-417f-aab0-2f78bab05fbd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:20.335653 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.335634    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b724a211-f022-417f-aab0-2f78bab05fbd-kube-api-access-5w595" pod "b724a211-f022-417f-aab0-2f78bab05fbd" (UID: "b724a211-f022-417f-aab0-2f78bab05fbd"). InnerVolumeSpecName "kube-api-access-5w595". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:20.430691 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.430655    5781 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b724a211-f022-417f-aab0-2f78bab05fbd-tls-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.430691 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.430692    5781 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b724a211-f022-417f-aab0-2f78bab05fbd-dshm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.430935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.430708    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w595\" (UniqueName: \"kubernetes.io/projected/b724a211-f022-417f-aab0-2f78bab05fbd-kube-api-access-5w595\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.451194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.451133    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main" containerID="cri-o://13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c" gracePeriod=41
Oct 30 13:19:20.707426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.707400    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:19:20.734464 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.734430    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tmp-dir\") pod \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") "
Oct 30 13:19:20.734610 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.734575    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-kube-api-access-dhbxl\" (UniqueName: \"kubernetes.io/projected/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-kube-api-access-dhbxl\") pod \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") "
Oct 30 13:19:20.734671 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.734632    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tls-certs\" (UniqueName: \"kubernetes.io/secret/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tls-certs\") pod \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") "
Oct 30 13:19:20.734721 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.734688    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-dshm\") pod \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") "
Oct 30 13:19:20.734721 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.734693    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tmp-dir" pod "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" (UID: "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:20.734843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.734720    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-home\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-home\") pod \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") "
Oct 30 13:19:20.734843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.734762    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-model-cache\") pod \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\" (UID: \"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b\") "
Oct 30 13:19:20.735053 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.735028    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-home" pod "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" (UID: "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:20.735192 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.735157    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-model-cache" pod "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" (UID: "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:20.735484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.735467    5781 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-home\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.735484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.735484    5781 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-model-cache\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.735614 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.735495    5781 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tmp-dir\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.736807 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.736789    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tls-certs" pod "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" (UID: "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:20.737065 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.737049    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-dshm" pod "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" (UID: "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:20.737112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.737055    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-kube-api-access-dhbxl" pod "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" (UID: "d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b"). InnerVolumeSpecName "kube-api-access-dhbxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:20.836706 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.836626    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhbxl\" (UniqueName: \"kubernetes.io/projected/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-kube-api-access-dhbxl\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.836706 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.836652    5781 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-tls-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:20.836706 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:20.836661    5781 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b-dshm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:21.013682 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.013642    5781 generic.go:356] "Generic (PLEG): container finished" podID="b724a211-f022-417f-aab0-2f78bab05fbd" containerID="7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81" exitCode=0
Oct 30 13:19:21.014112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.013694    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"
Oct 30 13:19:21.014112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.013729    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" event={"ID":"b724a211-f022-417f-aab0-2f78bab05fbd","Type":"ContainerDied","Data":"7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81"}
Oct 30 13:19:21.014112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.013773    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx" event={"ID":"b724a211-f022-417f-aab0-2f78bab05fbd","Type":"ContainerDied","Data":"46fda9eafbfe7b4a9cd321d489434c37b9500842424ff6bdb6c6b608ac61a004"}
Oct 30 13:19:21.014112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.013794    5781 scope.go:122] "RemoveContainer" containerID="7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81"
Oct 30 13:19:21.015553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.015526    5781 generic.go:356] "Generic (PLEG): container finished" podID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerID="13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c" exitCode=0
Oct 30 13:19:21.015660 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.015574    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"
Oct 30 13:19:21.015737 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.015562    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" event={"ID":"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b","Type":"ContainerDied","Data":"13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c"}
Oct 30 13:19:21.015737 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.015693    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8" event={"ID":"d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b","Type":"ContainerDied","Data":"9e003aeb1b5cd4d69a322e23deea99ba3dfce0c9b267c7be72d846d7e49fcb60"}
Oct 30 13:19:21.026808 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.026787    5781 scope.go:122] "RemoveContainer" containerID="7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81"
Oct 30 13:19:21.027102 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:21.027082    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81\": container with ID starting with 7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81 not found: ID does not exist" containerID="7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81"
Oct 30 13:19:21.027189 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.027108    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81"} err="failed to get container status \"7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81\": rpc error: code = NotFound desc = could not find container \"7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81\": container with ID starting with 7f7e5e6f84dd99af87d60ded58cced8e3549511d7a7cf219afe95575827a5e81 not found: ID does not exist"
Oct 30 13:19:21.027189 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.027125    5781 scope.go:122] "RemoveContainer" containerID="13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c"
Oct 30 13:19:21.044820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.044174    5781 scope.go:122] "RemoveContainer" containerID="13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c"
Oct 30 13:19:21.044820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.044588    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:21.045612 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:21.045582    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c\": container with ID starting with 13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c not found: ID does not exist" containerID="13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c"
Oct 30 13:19:21.045719 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.045621    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c"} err="failed to get container status \"13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c\": rpc error: code = NotFound desc = could not find container \"13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c\": container with ID starting with 13cbb9c4aab5c3239c42969733372de058070668c70853e986fed68726c3bd5c not found: ID does not exist"
Oct 30 13:19:21.073783 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.073755    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"]
Oct 30 13:19:21.081674 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.081644    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/simulator-w5-f8a55e-kserve-564878cb54-tvgk8"]
Oct 30 13:19:21.130444 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.130401    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"]
Oct 30 13:19:21.137250 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.137218    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-f8a55e/distinct-2-w5-f8a55e-kserve-6f5bdcbdb5-v6pxx"]
Oct 30 13:19:21.216772 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.216745    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:21.243074 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.243044    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-openshift-ca\") pod \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") "
Oct 30 13:19:21.243281 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.243266    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-xks-ca\") pod \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") "
Oct 30 13:19:21.243389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.243370    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-openshift-ca" pod "63254bd2-cb27-4bd6-a4c4-5562a9d6262c" (UID: "63254bd2-cb27-4bd6-a4c4-5562a9d6262c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:21.243430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.243392    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-kube-api-access-c7r22\" (UniqueName: \"kubernetes.io/projected/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-kube-api-access-c7r22\") pod \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\" (UID: \"63254bd2-cb27-4bd6-a4c4-5562a9d6262c\") "
Oct 30 13:19:21.244041 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.244016    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:21.245280 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.245254    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-xks-ca" pod "63254bd2-cb27-4bd6-a4c4-5562a9d6262c" (UID: "63254bd2-cb27-4bd6-a4c4-5562a9d6262c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:21.245395 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.245333    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-kube-api-access-c7r22" pod "63254bd2-cb27-4bd6-a4c4-5562a9d6262c" (UID: "63254bd2-cb27-4bd6-a4c4-5562a9d6262c"). InnerVolumeSpecName "kube-api-access-c7r22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:21.346026 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.345946    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:21.346026 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.345979    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7r22\" (UniqueName: \"kubernetes.io/projected/63254bd2-cb27-4bd6-a4c4-5562a9d6262c-kube-api-access-c7r22\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:21.993495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.993464    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"]
Oct 30 13:19:21.994414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.994393    5781 cpu_manager.go:414] "containerMap: removing container" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main"
Oct 30 13:19:21.994472 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.994422    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main"
Oct 30 13:19:21.994472 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.994448    5781 cpu_manager.go:414] "containerMap: removing container" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main"
Oct 30 13:19:21.994472 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.994456    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main"
Oct 30 13:19:21.994579 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.994483    5781 cpu_manager.go:414] "containerMap: removing container" podUID="63254bd2-cb27-4bd6-a4c4-5562a9d6262c" containerName="revoke-keys"
Oct 30 13:19:21.994579 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.994490    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63254bd2-cb27-4bd6-a4c4-5562a9d6262c" containerName="revoke-keys"
Oct 30 13:19:21.994636 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.994614    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="63254bd2-cb27-4bd6-a4c4-5562a9d6262c" containerName="revoke-keys"
Oct 30 13:19:21.994636 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.994631    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" containerName="main"
Oct 30 13:19:21.994699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.994637    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" containerName="main"
Oct 30 13:19:21.998566 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:21.998538    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.022233 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.022203    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"]
Oct 30 13:19:22.026571 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.026548    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-f76c09\"/\"distinct-w2-f76c09-kserve-self-signed-certs\""
Oct 30 13:19:22.030010 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.029988    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"
Oct 30 13:19:22.030131 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.030015    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf" event={"ID":"63254bd2-cb27-4bd6-a4c4-5562a9d6262c","Type":"ContainerDied","Data":"2200569ad9f7277aafde80f89548f07d8752e6dc8506e3ddc0233b03ffd2c025"}
Oct 30 13:19:22.030131 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.030067    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2200569ad9f7277aafde80f89548f07d8752e6dc8506e3ddc0233b03ffd2c025"
Oct 30 13:19:22.056766 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.056734    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-dshm\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.056962 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.056805    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4969603-8e31-49dd-ada8-ac59276f30b2-tls-certs\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.057087 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.057057    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfmrr\" (UniqueName: \"kubernetes.io/projected/e4969603-8e31-49dd-ada8-ac59276f30b2-kube-api-access-pfmrr\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.057194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.057113    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-tmp-dir\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.057263 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.057220    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-home\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.057353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.057333    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-model-cache\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.121013 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.120980    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:22.121718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.121698    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"]
Oct 30 13:19:22.126193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.126178    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.142181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.142160    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:22.142433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.142409    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:22.150857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.150832    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"]
Oct 30 13:19:22.158710 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.158684    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfmrr\" (UniqueName: \"kubernetes.io/projected/e4969603-8e31-49dd-ada8-ac59276f30b2-kube-api-access-pfmrr\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.158868 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.158723    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-tmp-dir\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.159013 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.158990    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-home\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.159089 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.159043    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-tmp-dir\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.159089 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.159082    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-be6b5d15-dlg56\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.159190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.159117    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-model-cache\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.159274 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.159253    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-be6b5d15-dlg56\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.159434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.159412    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-home\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.159512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.159434    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-model-cache\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.159512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.159485    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b77k6\" (UniqueName: \"kubernetes.io/projected/dc4cdbe5-257b-4f88-a491-21745923343b-kube-api-access-b77k6\") pod \"maas-api-revoke-sub-be6b5d15-dlg56\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.159617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.159534    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-dshm\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.159671 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.159611    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4969603-8e31-49dd-ada8-ac59276f30b2-tls-certs\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.162391 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.162369    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-dshm\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.165829 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.165807    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4969603-8e31-49dd-ada8-ac59276f30b2-tls-certs\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.182052 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.182022    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfmrr\" (UniqueName: \"kubernetes.io/projected/e4969603-8e31-49dd-ada8-ac59276f30b2-kube-api-access-pfmrr\") pod \"distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") " pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.261193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.261103    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-be6b5d15-dlg56\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.261193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.261166    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-be6b5d15-dlg56\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.261463 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.261233    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b77k6\" (UniqueName: \"kubernetes.io/projected/dc4cdbe5-257b-4f88-a491-21745923343b-kube-api-access-b77k6\") pod \"maas-api-revoke-sub-be6b5d15-dlg56\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.261919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.261897    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-be6b5d15-dlg56\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.263607 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.263579    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-be6b5d15-dlg56\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.286201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.286172    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b77k6\" (UniqueName: \"kubernetes.io/projected/dc4cdbe5-257b-4f88-a491-21745923343b-kube-api-access-b77k6\") pod \"maas-api-revoke-sub-be6b5d15-dlg56\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.313307 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.313262    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:22.443372 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.441900    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:22.654561 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.654529    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"]
Oct 30 13:19:22.655777 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:19:22.655748    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4969603_8e31_49dd_ada8_ac59276f30b2.slice/crio-983a24a3136fdd073ca14a47cf57a144f862e497b77fe70ba1f72d9acc87ce2c WatchSource:0}: Error finding container 983a24a3136fdd073ca14a47cf57a144f862e497b77fe70ba1f72d9acc87ce2c: Status 404 returned error can't find the container with id 983a24a3136fdd073ca14a47cf57a144f862e497b77fe70ba1f72d9acc87ce2c
Oct 30 13:19:22.761324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.761283    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"]
Oct 30 13:19:22.763519 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:19:22.763457    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc4cdbe5_257b_4f88_a491_21745923343b.slice/crio-c6fe01e712b8ae0ac934dd02524a0208a7d6f6a9f2a4ad36cd95a9896c1ef5de WatchSource:0}: Error finding container c6fe01e712b8ae0ac934dd02524a0208a7d6f6a9f2a4ad36cd95a9896c1ef5de: Status 404 returned error can't find the container with id c6fe01e712b8ae0ac934dd02524a0208a7d6f6a9f2a4ad36cd95a9896c1ef5de
Oct 30 13:19:22.843036 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.843007    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b724a211-f022-417f-aab0-2f78bab05fbd" path="/var/lib/kubelet/pods/b724a211-f022-417f-aab0-2f78bab05fbd/volumes"
Oct 30 13:19:22.843534 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.843520    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b" path="/var/lib/kubelet/pods/d9685e2b-61e0-4b77-9d7f-fae0e89e5d0b/volumes"
Oct 30 13:19:22.902505 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:22.902472    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:23.039049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:23.039020    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56" event={"ID":"dc4cdbe5-257b-4f88-a491-21745923343b","Type":"ContainerStarted","Data":"71b05b87023678cd519198c124bfbf4c647e786b1e4d606b7e70fb5002ed6921"}
Oct 30 13:19:23.039653 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:23.039080    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56" event={"ID":"dc4cdbe5-257b-4f88-a491-21745923343b","Type":"ContainerStarted","Data":"c6fe01e712b8ae0ac934dd02524a0208a7d6f6a9f2a4ad36cd95a9896c1ef5de"}
Oct 30 13:19:23.040638 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:23.040611    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" event={"ID":"e4969603-8e31-49dd-ada8-ac59276f30b2","Type":"ContainerStarted","Data":"c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b"}
Oct 30 13:19:23.040738 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:23.040645    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" event={"ID":"e4969603-8e31-49dd-ada8-ac59276f30b2","Type":"ContainerStarted","Data":"983a24a3136fdd073ca14a47cf57a144f862e497b77fe70ba1f72d9acc87ce2c"}
Oct 30 13:19:23.090146 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:23.090102    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56" podStartSLOduration=1.090087215 podStartE2EDuration="1.090087215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:23.090086903 +0000 UTC m=+1300.387304455" watchObservedRunningTime="2026-10-30 13:19:23.090087215 +0000 UTC m=+1300.387304757"
Oct 30 13:19:23.142384 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:23.142263    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podStartSLOduration=2.142251117 podStartE2EDuration="2.142251117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:23.140351713 +0000 UTC m=+1300.437569255" watchObservedRunningTime="2026-10-30 13:19:23.142251117 +0000 UTC m=+1300.439468660"
Oct 30 13:19:23.314062 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:23.313982    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:23.570771 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:23.570705    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"]
Oct 30 13:19:23.849398 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:23.849321    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:24.050076 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:24.050043    5781 generic.go:356] "Generic (PLEG): container finished" podID="dc4cdbe5-257b-4f88-a491-21745923343b" containerID="71b05b87023678cd519198c124bfbf4c647e786b1e4d606b7e70fb5002ed6921" exitCode=0
Oct 30 13:19:24.050491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:24.050134    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56" event={"ID":"dc4cdbe5-257b-4f88-a491-21745923343b","Type":"ContainerDied","Data":"71b05b87023678cd519198c124bfbf4c647e786b1e4d606b7e70fb5002ed6921"}
Oct 30 13:19:24.487966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:24.487931    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:25.217818 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.217792    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:25.313623 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.313538    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc4cdbe5-257b-4f88-a491-21745923343b-kube-api-access-b77k6\" (UniqueName: \"kubernetes.io/projected/dc4cdbe5-257b-4f88-a491-21745923343b-kube-api-access-b77k6\") pod \"dc4cdbe5-257b-4f88-a491-21745923343b\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") "
Oct 30 13:19:25.313846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.313821    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-xks-ca\") pod \"dc4cdbe5-257b-4f88-a491-21745923343b\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") "
Oct 30 13:19:25.314006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.313900    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-openshift-ca\") pod \"dc4cdbe5-257b-4f88-a491-21745923343b\" (UID: \"dc4cdbe5-257b-4f88-a491-21745923343b\") "
Oct 30 13:19:25.314321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.314248    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-openshift-ca" pod "dc4cdbe5-257b-4f88-a491-21745923343b" (UID: "dc4cdbe5-257b-4f88-a491-21745923343b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:25.314617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.314594    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:25.315774 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.315743    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc4cdbe5-257b-4f88-a491-21745923343b-kube-api-access-b77k6" pod "dc4cdbe5-257b-4f88-a491-21745923343b" (UID: "dc4cdbe5-257b-4f88-a491-21745923343b"). InnerVolumeSpecName "kube-api-access-b77k6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:25.316255 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.316229    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-xks-ca" pod "dc4cdbe5-257b-4f88-a491-21745923343b" (UID: "dc4cdbe5-257b-4f88-a491-21745923343b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:25.416637 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.416615    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc4cdbe5-257b-4f88-a491-21745923343b-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:25.416637 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:25.416638    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b77k6\" (UniqueName: \"kubernetes.io/projected/dc4cdbe5-257b-4f88-a491-21745923343b-kube-api-access-b77k6\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:26.071378 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:26.071331    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56" event={"ID":"dc4cdbe5-257b-4f88-a491-21745923343b","Type":"ContainerDied","Data":"c6fe01e712b8ae0ac934dd02524a0208a7d6f6a9f2a4ad36cd95a9896c1ef5de"}
Oct 30 13:19:26.071552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:26.071432    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6fe01e712b8ae0ac934dd02524a0208a7d6f6a9f2a4ad36cd95a9896c1ef5de"
Oct 30 13:19:26.071552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:26.071522    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"
Oct 30 13:19:26.186602 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:26.186569    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:26.729880 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:26.729850    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:28.096402 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:28.096365    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx" podUID="4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:28.181809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:28.181754    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:29.043737 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:29.043707    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"]
Oct 30 13:19:29.204337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:29.204237    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:29.386983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:29.386885    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:29.989600 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:29.989569    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:31.386232 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:31.386198    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:31.576455 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:31.576423    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:32.056915 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.053105    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:19:32.108706 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:32.108669    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:19:32.119150 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.118193    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:32.128039 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.128005    5781 generic.go:356] "Generic (PLEG): container finished" podID="4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" containerID="09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1" exitCode=0
Oct 30 13:19:32.128194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.128056    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"
Oct 30 13:19:32.128194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.128082    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx" event={"ID":"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7","Type":"ContainerDied","Data":"09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1"}
Oct 30 13:19:32.128194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.128161    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx" event={"ID":"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7","Type":"ContainerDied","Data":"4e79f687ae999c881276c831ba5bbefcd2a680ca910433a078a85ba1dd56e21f"}
Oct 30 13:19:32.128194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.128192    5781 scope.go:122] "RemoveContainer" containerID="09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1"
Oct 30 13:19:32.133559 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.133532    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-kube-api-access-j8jvj\" (UniqueName: \"kubernetes.io/projected/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-kube-api-access-j8jvj\") pod \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\" (UID: \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\") "
Oct 30 13:19:32.133740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.133679    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-plugins-config-volume\") pod \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\" (UID: \"4b6da6ab-fcf2-433e-b12a-dcea9a1daff7\") "
Oct 30 13:19:32.134059 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.134026    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-plugins-config-volume" pod "4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" (UID: "4b6da6ab-fcf2-433e-b12a-dcea9a1daff7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:32.134381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.134353    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:32.140394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.140197    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-kube-api-access-j8jvj" pod "4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" (UID: "4b6da6ab-fcf2-433e-b12a-dcea9a1daff7"). InnerVolumeSpecName "kube-api-access-j8jvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:32.146552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.146481    5781 scope.go:122] "RemoveContainer" containerID="09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1"
Oct 30 13:19:32.147156 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:32.147066    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1\": container with ID starting with 09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1 not found: ID does not exist" containerID="09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1"
Oct 30 13:19:32.147156 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.147121    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1"} err="failed to get container status \"09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1\": rpc error: code = NotFound desc = could not find container \"09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1\": container with ID starting with 09d709554c167f2b5580f381617e3d834188519fbf639315dd1ed5ff66b27ac1 not found: ID does not exist"
Oct 30 13:19:32.235424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.235390    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8jvj\" (UniqueName: \"kubernetes.io/projected/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7-kube-api-access-j8jvj\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:32.294074 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.293745    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"]
Oct 30 13:19:32.303733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.303692    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w5-f8a55e-mfjgz"]
Oct 30 13:19:32.318491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.318459    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:32.332813 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.332782    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:32.388749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.388716    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:32.487933 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.487896    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"]
Oct 30 13:19:32.498391 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.498362    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-f8a55e-c454d7c9b-nfrpx"]
Oct 30 13:19:32.850276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.850237    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" path="/var/lib/kubelet/pods/4b6da6ab-fcf2-433e-b12a-dcea9a1daff7/volumes"
Oct 30 13:19:32.850736 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.850719    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3d96339-09a6-4e5e-aca4-af1b4b454fff" path="/var/lib/kubelet/pods/c3d96339-09a6-4e5e-aca4-af1b4b454fff/volumes"
Oct 30 13:19:32.855876 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:32.855846    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:33.147430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.147365    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:33.389497 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.389461    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:33.780703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.780672    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"]
Oct 30 13:19:33.781320 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.781281    5781 cpu_manager.go:414] "containerMap: removing container" podUID="4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" containerName="payload-processing"
Oct 30 13:19:33.781320 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.781315    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" containerName="payload-processing"
Oct 30 13:19:33.781402 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.781332    5781 cpu_manager.go:414] "containerMap: removing container" podUID="dc4cdbe5-257b-4f88-a491-21745923343b" containerName="revoke-keys"
Oct 30 13:19:33.781402 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.781337    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc4cdbe5-257b-4f88-a491-21745923343b" containerName="revoke-keys"
Oct 30 13:19:33.781476 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.781467    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b6da6ab-fcf2-433e-b12a-dcea9a1daff7" containerName="payload-processing"
Oct 30 13:19:33.781512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.781479    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc4cdbe5-257b-4f88-a491-21745923343b" containerName="revoke-keys"
Oct 30 13:19:33.785419 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.785400    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:33.811705 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.811668    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"]
Oct 30 13:19:33.814918 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.814894    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:33.815132 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.815114    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:33.861499 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.861469    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a11dbc35-pksgl\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:33.861787 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.861769    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29c9x\" (UniqueName: \"kubernetes.io/projected/cd44d6d9-e84d-4823-adbe-51513dffde59-kube-api-access-29c9x\") pod \"maas-api-revoke-sub-a11dbc35-pksgl\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:33.861916 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.861900    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a11dbc35-pksgl\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:33.962911 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.962883    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29c9x\" (UniqueName: \"kubernetes.io/projected/cd44d6d9-e84d-4823-adbe-51513dffde59-kube-api-access-29c9x\") pod \"maas-api-revoke-sub-a11dbc35-pksgl\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:33.963049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.962964    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a11dbc35-pksgl\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:33.963111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.963052    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a11dbc35-pksgl\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:33.963742 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.963718    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a11dbc35-pksgl\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:33.965625 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.965602    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a11dbc35-pksgl\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:33.985760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:33.985740    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29c9x\" (UniqueName: \"kubernetes.io/projected/cd44d6d9-e84d-4823-adbe-51513dffde59-kube-api-access-29c9x\") pod \"maas-api-revoke-sub-a11dbc35-pksgl\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:34.044545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:34.044492    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r" podUID="7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" containerName="payload-pre-processing" containerID="cri-o://3d451cb5cca7852ed1daa13df4001962c1cc8a3a575f72f36b1590191fc58864" gracePeriod=25
Oct 30 13:19:34.100724 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:34.100697    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:34.181375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:34.177109    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:34.427338 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:34.427305    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"]
Oct 30 13:19:34.430750 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:19:34.430717    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd44d6d9_e84d_4823_adbe_51513dffde59.slice/crio-c28b6f673b7a183b714f4ed344bf20d406f6c8daf5344c6bf208b4cdbb429161 WatchSource:0}: Error finding container c28b6f673b7a183b714f4ed344bf20d406f6c8daf5344c6bf208b4cdbb429161: Status 404 returned error can't find the container with id c28b6f673b7a183b714f4ed344bf20d406f6c8daf5344c6bf208b4cdbb429161
Oct 30 13:19:35.054341 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:35.054268    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:35.155592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:35.155559    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl" event={"ID":"cd44d6d9-e84d-4823-adbe-51513dffde59","Type":"ContainerStarted","Data":"5eda19b1106a3403b280cbd9d81fe0da8e818417d4b2b5abdd6b0425b99c4b7e"}
Oct 30 13:19:35.155746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:35.155617    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl" event={"ID":"cd44d6d9-e84d-4823-adbe-51513dffde59","Type":"ContainerStarted","Data":"c28b6f673b7a183b714f4ed344bf20d406f6c8daf5344c6bf208b4cdbb429161"}
Oct 30 13:19:35.196961 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:35.196906    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl" podStartSLOduration=2.19689443 podStartE2EDuration="2.19689443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:35.196212916 +0000 UTC m=+1312.493430458" watchObservedRunningTime="2026-10-30 13:19:35.19689443 +0000 UTC m=+1312.494111971"
Oct 30 13:19:35.800094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:35.800063    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:35.999402 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:35.999363    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r" podUID="7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:36.165170 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:36.165083    5781 generic.go:356] "Generic (PLEG): container finished" podID="cd44d6d9-e84d-4823-adbe-51513dffde59" containerID="5eda19b1106a3403b280cbd9d81fe0da8e818417d4b2b5abdd6b0425b99c4b7e" exitCode=0
Oct 30 13:19:36.165354 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:36.165166    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl" event={"ID":"cd44d6d9-e84d-4823-adbe-51513dffde59","Type":"ContainerDied","Data":"5eda19b1106a3403b280cbd9d81fe0da8e818417d4b2b5abdd6b0425b99c4b7e"}
Oct 30 13:19:37.329197 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.329166    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:37.409532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.409504    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-openshift-ca\") pod \"cd44d6d9-e84d-4823-adbe-51513dffde59\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") "
Oct 30 13:19:37.409694 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.409586    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-xks-ca\") pod \"cd44d6d9-e84d-4823-adbe-51513dffde59\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") "
Oct 30 13:19:37.409775 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.409725    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd44d6d9-e84d-4823-adbe-51513dffde59-kube-api-access-29c9x\" (UniqueName: \"kubernetes.io/projected/cd44d6d9-e84d-4823-adbe-51513dffde59-kube-api-access-29c9x\") pod \"cd44d6d9-e84d-4823-adbe-51513dffde59\" (UID: \"cd44d6d9-e84d-4823-adbe-51513dffde59\") "
Oct 30 13:19:37.409952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.409910    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-openshift-ca" pod "cd44d6d9-e84d-4823-adbe-51513dffde59" (UID: "cd44d6d9-e84d-4823-adbe-51513dffde59"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:37.410369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.410336    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:37.411651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.411616    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-xks-ca" pod "cd44d6d9-e84d-4823-adbe-51513dffde59" (UID: "cd44d6d9-e84d-4823-adbe-51513dffde59"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:37.412456 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.412421    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd44d6d9-e84d-4823-adbe-51513dffde59-kube-api-access-29c9x" pod "cd44d6d9-e84d-4823-adbe-51513dffde59" (UID: "cd44d6d9-e84d-4823-adbe-51513dffde59"). InnerVolumeSpecName "kube-api-access-29c9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:37.511813 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.511743    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cd44d6d9-e84d-4823-adbe-51513dffde59-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:37.511813 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.511769    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29c9x\" (UniqueName: \"kubernetes.io/projected/cd44d6d9-e84d-4823-adbe-51513dffde59-kube-api-access-29c9x\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:37.586114 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:37.586070    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:38.183733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:38.183707    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"
Oct 30 13:19:38.183912 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:38.183749    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl" event={"ID":"cd44d6d9-e84d-4823-adbe-51513dffde59","Type":"ContainerDied","Data":"c28b6f673b7a183b714f4ed344bf20d406f6c8daf5344c6bf208b4cdbb429161"}
Oct 30 13:19:38.183912 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:38.183795    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c28b6f673b7a183b714f4ed344bf20d406f6c8daf5344c6bf208b4cdbb429161"
Oct 30 13:19:38.425470 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:38.425433    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"]
Oct 30 13:19:38.433594 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:38.433554    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"]
Oct 30 13:19:38.598865 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:38.598705    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" containerID="cri-o://b7b3f497bb06c8e30fa32aa212119682dff2b47997e018f8799556845b9db1ca" gracePeriod=45
Oct 30 13:19:38.604908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:38.604872    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:38.611760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:38.610531    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": EOF"
Oct 30 13:19:39.391572 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:39.391489    5781 generic.go:356] "Generic (PLEG): container finished" podID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerID="b7b3f497bb06c8e30fa32aa212119682dff2b47997e018f8799556845b9db1ca" exitCode=0
Oct 30 13:19:39.406861 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:39.393704    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" event={"ID":"b3f83e9b-225d-46a0-9976-ebedbc846a72","Type":"ContainerDied","Data":"b7b3f497bb06c8e30fa32aa212119682dff2b47997e018f8799556845b9db1ca"}
Oct 30 13:19:40.039313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.037588    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:19:40.152366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.151464    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:40.179310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.167758    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-dshm\") pod \"b3f83e9b-225d-46a0-9976-ebedbc846a72\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") "
Oct 30 13:19:40.179310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.167900    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3f83e9b-225d-46a0-9976-ebedbc846a72-kube-api-access-zbxgd\" (UniqueName: \"kubernetes.io/projected/b3f83e9b-225d-46a0-9976-ebedbc846a72-kube-api-access-zbxgd\") pod \"b3f83e9b-225d-46a0-9976-ebedbc846a72\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") "
Oct 30 13:19:40.179310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.168094    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-model-cache\") pod \"b3f83e9b-225d-46a0-9976-ebedbc846a72\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") "
Oct 30 13:19:40.179310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.168137    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3f83e9b-225d-46a0-9976-ebedbc846a72-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f83e9b-225d-46a0-9976-ebedbc846a72-tls-certs\") pod \"b3f83e9b-225d-46a0-9976-ebedbc846a72\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") "
Oct 30 13:19:40.179310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.168186    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-tmp-dir\") pod \"b3f83e9b-225d-46a0-9976-ebedbc846a72\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") "
Oct 30 13:19:40.179310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.168338    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-home\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-home\") pod \"b3f83e9b-225d-46a0-9976-ebedbc846a72\" (UID: \"b3f83e9b-225d-46a0-9976-ebedbc846a72\") "
Oct 30 13:19:40.201308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.193337    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-tmp-dir" pod "b3f83e9b-225d-46a0-9976-ebedbc846a72" (UID: "b3f83e9b-225d-46a0-9976-ebedbc846a72"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:40.213180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.210507    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-home" pod "b3f83e9b-225d-46a0-9976-ebedbc846a72" (UID: "b3f83e9b-225d-46a0-9976-ebedbc846a72"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:40.311312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.295680    5781 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-tmp-dir\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:40.319408 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.319361    5781 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-home\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:40.348326 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.301653    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3f83e9b-225d-46a0-9976-ebedbc846a72-kube-api-access-zbxgd" pod "b3f83e9b-225d-46a0-9976-ebedbc846a72" (UID: "b3f83e9b-225d-46a0-9976-ebedbc846a72"). InnerVolumeSpecName "kube-api-access-zbxgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:40.566353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.525847    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:40.822150 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.795962    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": context deadline exceeded"
Oct 30 13:19:40.822150 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.818409    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:40.824427 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.823348    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 13:19:40.856312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.826186    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main" probeResult="failure" output="Get \"https://10.134.0.89:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:40.907313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.878333    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:19:40.919322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.911447    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-model-cache" pod "b3f83e9b-225d-46a0-9976-ebedbc846a72" (UID: "b3f83e9b-225d-46a0-9976-ebedbc846a72"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:40.919322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:40.916888    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbxgd\" (UniqueName: \"kubernetes.io/projected/b3f83e9b-225d-46a0-9976-ebedbc846a72-kube-api-access-zbxgd\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:41.044363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.028374    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:41.238314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.228250    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.273313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.260346    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:41.273313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.235588    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-dshm" pod "b3f83e9b-225d-46a0-9976-ebedbc846a72" (UID: "b3f83e9b-225d-46a0-9976-ebedbc846a72"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:41.273313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.171720    5781 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-model-cache\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:41.315327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.297035    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.315327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.235331    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.439429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.437106    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3f83e9b-225d-46a0-9976-ebedbc846a72-tls-certs" pod "b3f83e9b-225d-46a0-9976-ebedbc846a72" (UID: "b3f83e9b-225d-46a0-9976-ebedbc846a72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:41.599399 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.579328    5781 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f83e9b-225d-46a0-9976-ebedbc846a72-tls-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:41.599399 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.579362    5781 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f83e9b-225d-46a0-9976-ebedbc846a72-dshm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:41.599399 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.579894    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.599399 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.531526    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:41.800315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.796305    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:41.865372 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.825468    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:42.297698 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:41.826383    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:19:42.298268 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:41.975457    5781 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.097s"
Oct 30 13:19:42.298473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.298459    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:42.302468 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.302435    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:19:42.318884 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.287494    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:42.319526 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.279817    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:42.694567 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.297554    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:42.703811 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.153479    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:19:42.704103 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.704055    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:42.714325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.319339    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:19:42.714325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.713085    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz" event={"ID":"b3f83e9b-225d-46a0-9976-ebedbc846a72","Type":"ContainerDied","Data":"25a2d525c4fe5866f1b79fa4a5643bb15f78d210d51b7eb54b6a73ee1d8f893d"}
Oct 30 13:19:42.714325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.713157    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:19:42.714325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.713199    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:19:42.714325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.713234    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:19:42.714325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.564752    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:42.722341 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.637022    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:42.722341 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.715141    5781 scope.go:122] "RemoveContainer" containerID="b7b3f497bb06c8e30fa32aa212119682dff2b47997e018f8799556845b9db1ca"
Oct 30 13:19:42.766345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.140222    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"
Oct 30 13:19:42.894550 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:42.894511    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.060312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.034750    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:43.433323 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.343557    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 13:19:43.590436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.546025    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.590436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.570921    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.703871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.703830    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.704133 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.704113    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/ready\": context deadline exceeded"
Oct 30 13:19:43.762310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.708271    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:43.799424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.787398    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": context deadline exceeded"
Oct 30 13:19:43.980981 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:43.980852    5781 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.008s"
Oct 30 13:19:43.981212 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.981192    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"]
Oct 30 13:19:43.981768 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.981747    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"]
Oct 30 13:19:43.985310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.983083    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy" containerID="cri-o://1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2" gracePeriod=30
Oct 30 13:19:43.991312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:43.989089    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy" containerID="cri-o://5a9ec9cb66a3ff780d31660c50cd629cfa2d897bc1ba60479bdbcbd98af0e482" gracePeriod=30
Oct 30 13:19:44.063473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.029268    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:19:44.086321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.085469    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:19:44.101315 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.098301    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:19:44.117223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.117197    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:19:44.178309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.174150    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:19:44.188361 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.183808    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"]
Oct 30 13:19:44.188361 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.183897    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:19:44.188361 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.184038    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" containerID="cri-o://f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d" gracePeriod=30
Oct 30 13:19:44.237264 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.237179    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:19:44.382625 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.382592    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"]
Oct 30 13:19:44.386451 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.386413    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-031737/distinct-2-w0-031737-kserve-6cbfc5fd5-xn4lz"]
Oct 30 13:19:44.619888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.619852    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:44.751189 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.751148    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:19:44.751378 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.751238    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:19:44.842945 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.842869    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" path="/var/lib/kubelet/pods/b3f83e9b-225d-46a0-9976-ebedbc846a72/volumes"
Oct 30 13:19:44.912530 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:44.912503    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"]
Oct 30 13:19:45.062209 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:45.062181    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:45.242371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:45.242343    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:45.393812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:45.393779    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:19:45.393996 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:45.393864    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:19:45.394678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:45.394654    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:45.984272 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:45.984244    5781 patch_prober.go:29] interesting pod/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:19:45.984627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:45.984301    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:19:45.999766 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:45.999720    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r" podUID="7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:46.369595 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:46.369478    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:48.276795 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.276760    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:48.649379 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.649258    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"]
Oct 30 13:19:48.650045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.650024    5781 cpu_manager.go:414] "containerMap: removing container" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main"
Oct 30 13:19:48.650045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.650046    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main"
Oct 30 13:19:48.650212 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.650104    5781 cpu_manager.go:414] "containerMap: removing container" podUID="cd44d6d9-e84d-4823-adbe-51513dffde59" containerName="revoke-keys"
Oct 30 13:19:48.650212 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.650113    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd44d6d9-e84d-4823-adbe-51513dffde59" containerName="revoke-keys"
Oct 30 13:19:48.650353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.650264    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd44d6d9-e84d-4823-adbe-51513dffde59" containerName="revoke-keys"
Oct 30 13:19:48.650353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.650300    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3f83e9b-225d-46a0-9976-ebedbc846a72" containerName="main"
Oct 30 13:19:48.655673 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.655652    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.683847 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.683818    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:19:48.683982 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.683867    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:19:48.688976 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.688950    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"]
Oct 30 13:19:48.777819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.777789    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f8d5f7cc-ssnvg\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.777943 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.777848    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f8d5f7cc-ssnvg\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.777997 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.777981    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxrcd\" (UniqueName: \"kubernetes.io/projected/e3e2b109-3318-4037-8d35-ad4aea11ce83-kube-api-access-dxrcd\") pod \"maas-api-revoke-sub-f8d5f7cc-ssnvg\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.879601 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.879572    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f8d5f7cc-ssnvg\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.879746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.879633    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxrcd\" (UniqueName: \"kubernetes.io/projected/e3e2b109-3318-4037-8d35-ad4aea11ce83-kube-api-access-dxrcd\") pod \"maas-api-revoke-sub-f8d5f7cc-ssnvg\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.879805 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.879789    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f8d5f7cc-ssnvg\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.880306 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.880267    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f8d5f7cc-ssnvg\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.882434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.882405    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f8d5f7cc-ssnvg\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.902068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.902014    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxrcd\" (UniqueName: \"kubernetes.io/projected/e3e2b109-3318-4037-8d35-ad4aea11ce83-kube-api-access-dxrcd\") pod \"maas-api-revoke-sub-f8d5f7cc-ssnvg\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:48.969644 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:48.969624    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:49.160743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.160688    5781 generic.go:356] "Generic (PLEG): container finished" podID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerID="5a9ec9cb66a3ff780d31660c50cd629cfa2d897bc1ba60479bdbcbd98af0e482" exitCode=0
Oct 30 13:19:49.160855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.160764    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" event={"ID":"9ebb19b0-d8c8-4f17-aa13-78f210c82265","Type":"ContainerDied","Data":"5a9ec9cb66a3ff780d31660c50cd629cfa2d897bc1ba60479bdbcbd98af0e482"}
Oct 30 13:19:49.278618 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.278493    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"]
Oct 30 13:19:49.297186 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:19:49.297153    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e2b109_3318_4037_8d35_ad4aea11ce83.slice/crio-7ee012fcd4b3402710257b5730cb6cea694c08b37805ec052703f69849f969a8 WatchSource:0}: Error finding container 7ee012fcd4b3402710257b5730cb6cea694c08b37805ec052703f69849f969a8: Status 404 returned error can't find the container with id 7ee012fcd4b3402710257b5730cb6cea694c08b37805ec052703f69849f969a8
Oct 30 13:19:49.533574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.533554    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:19:49.535666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.535646    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:19:49.589759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.589736    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-certs\") pod \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") "
Oct 30 13:19:49.589864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.589823    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-data\") pod \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") "
Oct 30 13:19:49.589905 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.589874    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-token\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-token\") pod \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") "
Oct 30 13:19:49.589939 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.589916    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-podinfo\") pod \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") "
Oct 30 13:19:49.590024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590002    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-socket\") pod \"6d198200-2f59-46b0-907d-d61d2c6cb378\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") "
Oct 30 13:19:49.590095 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590014    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-certs" pod "76c0e318-a41a-4edb-b0e6-9652cf048d9a" (UID: "76c0e318-a41a-4edb-b0e6-9652cf048d9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.590095 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590070    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-data" pod "76c0e318-a41a-4edb-b0e6-9652cf048d9a" (UID: "76c0e318-a41a-4edb-b0e6-9652cf048d9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.590202 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590103    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istiod-ca-cert\") pod \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") "
Oct 30 13:19:49.590202 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590129    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d198200-2f59-46b0-907d-d61d2c6cb378-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d198200-2f59-46b0-907d-d61d2c6cb378-istiod-ca-cert\") pod \"6d198200-2f59-46b0-907d-d61d2c6cb378\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") "
Oct 30 13:19:49.590202 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590172    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-certs\") pod \"6d198200-2f59-46b0-907d-d61d2c6cb378\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") "
Oct 30 13:19:49.590397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590200    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-kube-api-access-zz2hf\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-kube-api-access-zz2hf\") pod \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") "
Oct 30 13:19:49.590397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590227    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-data\") pod \"6d198200-2f59-46b0-907d-d61d2c6cb378\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") "
Oct 30 13:19:49.590397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590264    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-socket\") pod \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") "
Oct 30 13:19:49.590397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590307    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-envoy\") pod \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") "
Oct 30 13:19:49.590397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590348    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-credential-socket\") pod \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\" (UID: \"76c0e318-a41a-4edb-b0e6-9652cf048d9a\") "
Oct 30 13:19:49.590397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590389    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-token\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-token\") pod \"6d198200-2f59-46b0-907d-d61d2c6cb378\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") "
Oct 30 13:19:49.590688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590434    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istiod-ca-cert" pod "76c0e318-a41a-4edb-b0e6-9652cf048d9a" (UID: "76c0e318-a41a-4edb-b0e6-9652cf048d9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:49.590688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590524    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-data" pod "6d198200-2f59-46b0-907d-d61d2c6cb378" (UID: "6d198200-2f59-46b0-907d-d61d2c6cb378"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.590688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590474    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-credential-socket\") pod \"6d198200-2f59-46b0-907d-d61d2c6cb378\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") "
Oct 30 13:19:49.590688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590593    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-envoy\") pod \"6d198200-2f59-46b0-907d-d61d2c6cb378\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") "
Oct 30 13:19:49.590688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590668    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-kube-api-access-56lfx\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-kube-api-access-56lfx\") pod \"6d198200-2f59-46b0-907d-d61d2c6cb378\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") "
Oct 30 13:19:49.590946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590709    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-podinfo\") pod \"6d198200-2f59-46b0-907d-d61d2c6cb378\" (UID: \"6d198200-2f59-46b0-907d-d61d2c6cb378\") "
Oct 30 13:19:49.590946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590539    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d198200-2f59-46b0-907d-d61d2c6cb378-istiod-ca-cert" pod "6d198200-2f59-46b0-907d-d61d2c6cb378" (UID: "6d198200-2f59-46b0-907d-d61d2c6cb378"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:49.590946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590646    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-socket" pod "6d198200-2f59-46b0-907d-d61d2c6cb378" (UID: "6d198200-2f59-46b0-907d-d61d2c6cb378"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.590946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590660    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-credential-socket" pod "6d198200-2f59-46b0-907d-d61d2c6cb378" (UID: "6d198200-2f59-46b0-907d-d61d2c6cb378"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.590946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590759    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-certs" pod "6d198200-2f59-46b0-907d-d61d2c6cb378" (UID: "6d198200-2f59-46b0-907d-d61d2c6cb378"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.590946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.590848    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-socket" pod "76c0e318-a41a-4edb-b0e6-9652cf048d9a" (UID: "76c0e318-a41a-4edb-b0e6-9652cf048d9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.591321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591150    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-credential-socket" pod "76c0e318-a41a-4edb-b0e6-9652cf048d9a" (UID: "76c0e318-a41a-4edb-b0e6-9652cf048d9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591464    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591490    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591506    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591519    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591533    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591548    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d198200-2f59-46b0-907d-d61d2c6cb378-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591560    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591572    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591585    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.591621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.591600    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.592907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.592883    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-podinfo" pod "76c0e318-a41a-4edb-b0e6-9652cf048d9a" (UID: "76c0e318-a41a-4edb-b0e6-9652cf048d9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:19:49.593017 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.592904    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-token" pod "76c0e318-a41a-4edb-b0e6-9652cf048d9a" (UID: "76c0e318-a41a-4edb-b0e6-9652cf048d9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.593017 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.592940    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-kube-api-access-zz2hf" pod "76c0e318-a41a-4edb-b0e6-9652cf048d9a" (UID: "76c0e318-a41a-4edb-b0e6-9652cf048d9a"). InnerVolumeSpecName "kube-api-access-zz2hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.593322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.593297    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-token" pod "6d198200-2f59-46b0-907d-d61d2c6cb378" (UID: "6d198200-2f59-46b0-907d-d61d2c6cb378"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.593385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.593324    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-podinfo" pod "6d198200-2f59-46b0-907d-d61d2c6cb378" (UID: "6d198200-2f59-46b0-907d-d61d2c6cb378"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:19:49.593606 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.593592    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-envoy" pod "6d198200-2f59-46b0-907d-d61d2c6cb378" (UID: "6d198200-2f59-46b0-907d-d61d2c6cb378"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.593643 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.593612    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-kube-api-access-56lfx" pod "6d198200-2f59-46b0-907d-d61d2c6cb378" (UID: "6d198200-2f59-46b0-907d-d61d2c6cb378"). InnerVolumeSpecName "kube-api-access-56lfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.594083 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.594070    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-envoy" pod "76c0e318-a41a-4edb-b0e6-9652cf048d9a" (UID: "76c0e318-a41a-4edb-b0e6-9652cf048d9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.694180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.694096    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zz2hf\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-kube-api-access-zz2hf\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.694349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.694194    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.694349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.694209    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.694349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.694221    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.694349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.694234    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56lfx\" (UniqueName: \"kubernetes.io/projected/6d198200-2f59-46b0-907d-d61d2c6cb378-kube-api-access-56lfx\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.694349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.694247    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d198200-2f59-46b0-907d-d61d2c6cb378-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.694349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.694259    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.694349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.694274    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76c0e318-a41a-4edb-b0e6-9652cf048d9a-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.726353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.725103    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"]
Oct 30 13:19:49.726353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.726092    5781 cpu_manager.go:414] "containerMap: removing container" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy"
Oct 30 13:19:49.726353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.726112    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy"
Oct 30 13:19:49.726353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.726209    5781 cpu_manager.go:414] "containerMap: removing container" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy"
Oct 30 13:19:49.726353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.726217    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy"
Oct 30 13:19:49.726663 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.726380    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerName="istio-proxy"
Oct 30 13:19:49.726663 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.726395    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerName="istio-proxy"
Oct 30 13:19:49.743760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.743663    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.763440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.762955    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"]
Oct 30 13:19:49.768742 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.768686    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 13:19:49.768848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.768813    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 13:19:49.768906 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.768890    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 13:19:49.769014 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.769002    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6ngqs\""
Oct 30 13:19:49.795141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.795114    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.795276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.795167    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.795276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.795226    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.795276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.795250    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.795535 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.795338    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.795535 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.795392    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p54jv\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-kube-api-access-p54jv\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.795535 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.795464    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.797715 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.797692    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:19:49.891502 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.891464    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:49.914333 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.914278    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz" podUID="7c951714-8bf1-4df2-af09-2c33656636d6" containerName="payload-pre-processing" containerID="cri-o://2edd0f5d4bc95c1b64ea31e3bac23ec3f707ab2c55074287ac411c91134d8494" gracePeriod=25
Oct 30 13:19:49.917218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.917196    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-token\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-token\") pod \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") "
Oct 30 13:19:49.917473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.917454    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-socket\") pod \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") "
Oct 30 13:19:49.917919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.917900    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istiod-ca-cert\") pod \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") "
Oct 30 13:19:49.918106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.918076    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-podinfo\") pod \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") "
Oct 30 13:19:49.918501 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.918283    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-credential-socket\") pod \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") "
Oct 30 13:19:49.918750 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.918731    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-data\") pod \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") "
Oct 30 13:19:49.918871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.918756    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-kube-api-access-df5jl\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-kube-api-access-df5jl\") pod \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") "
Oct 30 13:19:49.918938 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.918884    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-certs\") pod \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") "
Oct 30 13:19:49.918938 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.918910    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-envoy\") pod \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\" (UID: \"9ebb19b0-d8c8-4f17-aa13-78f210c82265\") "
Oct 30 13:19:49.929472 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.922092    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-socket" pod "9ebb19b0-d8c8-4f17-aa13-78f210c82265" (UID: "9ebb19b0-d8c8-4f17-aa13-78f210c82265"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.929472 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.928836    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-data" pod "9ebb19b0-d8c8-4f17-aa13-78f210c82265" (UID: "9ebb19b0-d8c8-4f17-aa13-78f210c82265"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.929472 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.929149    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-credential-socket" pod "9ebb19b0-d8c8-4f17-aa13-78f210c82265" (UID: "9ebb19b0-d8c8-4f17-aa13-78f210c82265"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.930332 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.930265    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istiod-ca-cert" pod "9ebb19b0-d8c8-4f17-aa13-78f210c82265" (UID: "9ebb19b0-d8c8-4f17-aa13-78f210c82265"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:49.930778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.930742    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-certs" pod "9ebb19b0-d8c8-4f17-aa13-78f210c82265" (UID: "9ebb19b0-d8c8-4f17-aa13-78f210c82265"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.931276    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.931423    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.931468    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.931598    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.931693    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p54jv\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-kube-api-access-p54jv\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.931809    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.931915    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.932042    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.932059    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.932074    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.932088    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.932331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.932102    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:49.942556 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.942523    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-token" pod "9ebb19b0-d8c8-4f17-aa13-78f210c82265" (UID: "9ebb19b0-d8c8-4f17-aa13-78f210c82265"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.942697 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.942571    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-kube-api-access-df5jl" pod "9ebb19b0-d8c8-4f17-aa13-78f210c82265" (UID: "9ebb19b0-d8c8-4f17-aa13-78f210c82265"). InnerVolumeSpecName "kube-api-access-df5jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:49.942775 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.942717    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-envoy" pod "9ebb19b0-d8c8-4f17-aa13-78f210c82265" (UID: "9ebb19b0-d8c8-4f17-aa13-78f210c82265"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:49.943786 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.943735    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.944776 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.944757    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-podinfo" pod "9ebb19b0-d8c8-4f17-aa13-78f210c82265" (UID: "9ebb19b0-d8c8-4f17-aa13-78f210c82265"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:19:49.945631 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.945576    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.946166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.946146    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.946587 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.946562    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.946827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.946806    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.956922 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.956677    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:49.959028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:49.959002    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p54jv\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-kube-api-access-p54jv\") pod \"istiod-openshift-gateway-7ddd9cfdc8-p28dw\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:50.033251 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.033189    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.033251 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.033213    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-df5jl\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-kube-api-access-df5jl\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.033251 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.033231    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.033251 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.033243    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ebb19b0-d8c8-4f17-aa13-78f210c82265-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:50.067445 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.067417    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:50.172173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.172137    5781 generic.go:356] "Generic (PLEG): container finished" podID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" containerID="f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d" exitCode=0
Oct 30 13:19:50.172354 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.172168    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" event={"ID":"76c0e318-a41a-4edb-b0e6-9652cf048d9a","Type":"ContainerDied","Data":"f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d"}
Oct 30 13:19:50.172354 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.172205    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"
Oct 30 13:19:50.172354 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.172235    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4" event={"ID":"76c0e318-a41a-4edb-b0e6-9652cf048d9a","Type":"ContainerDied","Data":"4038fe1d82d02a5d45e4076c0843f8199bad4bad3b1807c5fbbda2fb067b14fd"}
Oct 30 13:19:50.172354 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.172260    5781 scope.go:122] "RemoveContainer" containerID="f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d"
Oct 30 13:19:50.177092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.176581    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg" event={"ID":"e3e2b109-3318-4037-8d35-ad4aea11ce83","Type":"ContainerStarted","Data":"c8a900b0ce9471ed6ac62732108316e5f088b002c5f6a9e44fa71563d75e4983"}
Oct 30 13:19:50.177092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.176630    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg" event={"ID":"e3e2b109-3318-4037-8d35-ad4aea11ce83","Type":"ContainerStarted","Data":"7ee012fcd4b3402710257b5730cb6cea694c08b37805ec052703f69849f969a8"}
Oct 30 13:19:50.178276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.178247    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"
Oct 30 13:19:50.178421 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.178303    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp" event={"ID":"9ebb19b0-d8c8-4f17-aa13-78f210c82265","Type":"ContainerDied","Data":"b6acf0475e11ecd2a0e2e4712d54d2907a2b09806ad0abc688ba2bd34f6ba7b8"}
Oct 30 13:19:50.180376 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.180350    5781 generic.go:356] "Generic (PLEG): container finished" podID="6d198200-2f59-46b0-907d-d61d2c6cb378" containerID="1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2" exitCode=0
Oct 30 13:19:50.180477 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.180385    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" event={"ID":"6d198200-2f59-46b0-907d-d61d2c6cb378","Type":"ContainerDied","Data":"1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2"}
Oct 30 13:19:50.180477 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.180412    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"
Oct 30 13:19:50.180559 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.180420    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm" event={"ID":"6d198200-2f59-46b0-907d-d61d2c6cb378","Type":"ContainerDied","Data":"e88fb2c7fd863496e9f6667c863e10ae760201f823e05ff7642e565bf08fadbe"}
Oct 30 13:19:50.190183 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.189851    5781 scope.go:122] "RemoveContainer" containerID="f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d"
Oct 30 13:19:50.190940 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:50.190913    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d\": container with ID starting with f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d not found: ID does not exist" containerID="f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d"
Oct 30 13:19:50.191015 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.190951    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d"} err="failed to get container status \"f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d\": rpc error: code = NotFound desc = could not find container \"f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d\": container with ID starting with f27066529e247255da2114ffc3de5ed72dee451d10e06b4ca2b9c3e9da19997d not found: ID does not exist"
Oct 30 13:19:50.191015 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.190975    5781 scope.go:122] "RemoveContainer" containerID="5a9ec9cb66a3ff780d31660c50cd629cfa2d897bc1ba60479bdbcbd98af0e482"
Oct 30 13:19:50.200751 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.200735    5781 scope.go:122] "RemoveContainer" containerID="1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2"
Oct 30 13:19:50.210391 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.210376    5781 scope.go:122] "RemoveContainer" containerID="1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2"
Oct 30 13:19:50.210645 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:50.210626    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2\": container with ID starting with 1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2 not found: ID does not exist" containerID="1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2"
Oct 30 13:19:50.210699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.210651    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2"} err="failed to get container status \"1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2\": rpc error: code = NotFound desc = could not find container \"1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2\": container with ID starting with 1b586d56d62cd2ac45f438f51b1c6bd0d81f62d81f3d6c3e21bc09ea689fc7d2 not found: ID does not exist"
Oct 30 13:19:50.245574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.245550    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"]
Oct 30 13:19:50.257138 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.257116    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-7s2r4"]
Oct 30 13:19:50.298811 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.298743    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg" podStartSLOduration=2.298731712 podStartE2EDuration="2.298731712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:50.296683646 +0000 UTC m=+1327.593901188" watchObservedRunningTime="2026-10-30 13:19:50.298731712 +0000 UTC m=+1327.595949255"
Oct 30 13:19:50.352424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.352401    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"]
Oct 30 13:19:50.360440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.360418    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-6sjkm"]
Oct 30 13:19:50.398633 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.398601    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"]
Oct 30 13:19:50.410143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.410106    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"]
Oct 30 13:19:50.411105 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:19:50.411083    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf914ca73_c724_4f0f_b59a_0a9c7fd8143d.slice/crio-c45c1826cdd268291fd91229b0698ff119f08d86502b3fcd26ddf62c26aae68d WatchSource:0}: Error finding container c45c1826cdd268291fd91229b0698ff119f08d86502b3fcd26ddf62c26aae68d: Status 404 returned error can't find the container with id c45c1826cdd268291fd91229b0698ff119f08d86502b3fcd26ddf62c26aae68d
Oct 30 13:19:50.413794 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.413759    5781 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:19:50.414670 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.414634    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-f8a55e-openshift-default-6cf69f7695-xpjfp"]
Oct 30 13:19:50.842377 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.842347    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d198200-2f59-46b0-907d-d61d2c6cb378" path="/var/lib/kubelet/pods/6d198200-2f59-46b0-907d-d61d2c6cb378/volumes"
Oct 30 13:19:50.843149 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.842839    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76c0e318-a41a-4edb-b0e6-9652cf048d9a" path="/var/lib/kubelet/pods/76c0e318-a41a-4edb-b0e6-9652cf048d9a/volumes"
Oct 30 13:19:50.843261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:50.843247    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" path="/var/lib/kubelet/pods/9ebb19b0-d8c8-4f17-aa13-78f210c82265/volumes"
Oct 30 13:19:51.050655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:51.050617    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz" podUID="7c951714-8bf1-4df2-af09-2c33656636d6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:19:51.176173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:51.176117    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:51.192093 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:51.192059    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw" event={"ID":"f914ca73-c724-4f0f-b59a-0a9c7fd8143d","Type":"ContainerStarted","Data":"c45c1826cdd268291fd91229b0698ff119f08d86502b3fcd26ddf62c26aae68d"}
Oct 30 13:19:51.195831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:51.195801    5781 generic.go:356] "Generic (PLEG): container finished" podID="e3e2b109-3318-4037-8d35-ad4aea11ce83" containerID="c8a900b0ce9471ed6ac62732108316e5f088b002c5f6a9e44fa71563d75e4983" exitCode=0
Oct 30 13:19:51.195955 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:51.195858    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg" event={"ID":"e3e2b109-3318-4037-8d35-ad4aea11ce83","Type":"ContainerDied","Data":"c8a900b0ce9471ed6ac62732108316e5f088b002c5f6a9e44fa71563d75e4983"}
Oct 30 13:19:51.625962 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:51.625875    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:52.399430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.399401    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:52.575792 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.575708    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-openshift-ca\") pod \"e3e2b109-3318-4037-8d35-ad4aea11ce83\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") "
Oct 30 13:19:52.575965 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.575817    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3e2b109-3318-4037-8d35-ad4aea11ce83-kube-api-access-dxrcd\" (UniqueName: \"kubernetes.io/projected/e3e2b109-3318-4037-8d35-ad4aea11ce83-kube-api-access-dxrcd\") pod \"e3e2b109-3318-4037-8d35-ad4aea11ce83\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") "
Oct 30 13:19:52.575965 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.575879    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-xks-ca\") pod \"e3e2b109-3318-4037-8d35-ad4aea11ce83\" (UID: \"e3e2b109-3318-4037-8d35-ad4aea11ce83\") "
Oct 30 13:19:52.576089 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.576072    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-openshift-ca" pod "e3e2b109-3318-4037-8d35-ad4aea11ce83" (UID: "e3e2b109-3318-4037-8d35-ad4aea11ce83"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:52.576608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.576574    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:52.583625 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.583600    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-xks-ca" pod "e3e2b109-3318-4037-8d35-ad4aea11ce83" (UID: "e3e2b109-3318-4037-8d35-ad4aea11ce83"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:52.585609 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.585586    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3e2b109-3318-4037-8d35-ad4aea11ce83-kube-api-access-dxrcd" pod "e3e2b109-3318-4037-8d35-ad4aea11ce83" (UID: "e3e2b109-3318-4037-8d35-ad4aea11ce83"). InnerVolumeSpecName "kube-api-access-dxrcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:52.678077 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.678044    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxrcd\" (UniqueName: \"kubernetes.io/projected/e3e2b109-3318-4037-8d35-ad4aea11ce83-kube-api-access-dxrcd\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:52.678077 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.678074    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e3e2b109-3318-4037-8d35-ad4aea11ce83-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:52.819545 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.819516    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:19:52.819631 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:52.819578    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:19:53.214071 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.214032    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw" event={"ID":"f914ca73-c724-4f0f-b59a-0a9c7fd8143d","Type":"ContainerStarted","Data":"136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484"}
Oct 30 13:19:53.214349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.214323    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:53.215539 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.215520    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"
Oct 30 13:19:53.215668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.215538    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg" event={"ID":"e3e2b109-3318-4037-8d35-ad4aea11ce83","Type":"ContainerDied","Data":"7ee012fcd4b3402710257b5730cb6cea694c08b37805ec052703f69849f969a8"}
Oct 30 13:19:53.215668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.215575    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ee012fcd4b3402710257b5730cb6cea694c08b37805ec052703f69849f969a8"
Oct 30 13:19:53.216418 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.216398    5781 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-p28dw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:19:53.216541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.216443    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw" podUID="f914ca73-c724-4f0f-b59a-0a9c7fd8143d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:19:53.277721 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.275923    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw" podStartSLOduration=1.871261682 podStartE2EDuration="4.27590507s" totalImagesPullingTime="2.404643388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:19:53.272383369 +0000 UTC m=+1330.569600911" watchObservedRunningTime="2026-10-30 13:19:53.27590507 +0000 UTC m=+1330.573122611"
Oct 30 13:19:53.332373 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.331630    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:53.474576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.474482    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" containerID="cri-o://716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7" gracePeriod=45
Oct 30 13:19:53.486570 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.486533    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main" probeResult="failure" output="Get \"https://10.134.0.58:8000/ready\": dial tcp 10.134.0.58:8000: connect: connection refused"
Oct 30 13:19:53.486887 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.486858    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main" containerID="cri-o://13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737" gracePeriod=45
Oct 30 13:19:53.771493 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.771465    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:19:53.818874 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.818851    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:19:53.896024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.895986    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-home\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-home\") pod \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") "
Oct 30 13:19:53.896229 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.896206    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tmp-dir\") pod \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") "
Oct 30 13:19:53.896329 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.896254    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tls-certs\" (UniqueName: \"kubernetes.io/secret/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tls-certs\") pod \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") "
Oct 30 13:19:53.896404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.896328    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-kube-api-access-4njjc\" (UniqueName: \"kubernetes.io/projected/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-kube-api-access-4njjc\") pod \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") "
Oct 30 13:19:53.896462 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.896402    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-dshm\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-dshm\") pod \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") "
Oct 30 13:19:53.896462 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.896442    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-model-cache\") pod \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\" (UID: \"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99\") "
Oct 30 13:19:53.897246 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.897214    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-home" pod "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" (UID: "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:53.897416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.897393    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-model-cache" pod "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" (UID: "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:53.897843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.897818    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tmp-dir" pod "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" (UID: "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:53.908165 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.908115    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tls-certs" pod "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" (UID: "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:53.909163 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.909122    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-dshm" pod "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" (UID: "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:53.913336 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.912854    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-kube-api-access-4njjc" pod "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" (UID: "520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99"). InnerVolumeSpecName "kube-api-access-4njjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:53.998492 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.998407    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-model-cache\") pod \"0ab619b4-ab20-4655-8def-7ed825afab2f\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") "
Oct 30 13:19:53.998628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.998564    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ab619b4-ab20-4655-8def-7ed825afab2f-kube-api-access-gqdqn\" (UniqueName: \"kubernetes.io/projected/0ab619b4-ab20-4655-8def-7ed825afab2f-kube-api-access-gqdqn\") pod \"0ab619b4-ab20-4655-8def-7ed825afab2f\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") "
Oct 30 13:19:53.998628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.998584    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-tmp-dir\") pod \"0ab619b4-ab20-4655-8def-7ed825afab2f\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") "
Oct 30 13:19:53.998728 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.998648    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-home\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-home\") pod \"0ab619b4-ab20-4655-8def-7ed825afab2f\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") "
Oct 30 13:19:53.998728 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.998674    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0ab619b4-ab20-4655-8def-7ed825afab2f-tls-certs\" (UniqueName: \"kubernetes.io/secret/0ab619b4-ab20-4655-8def-7ed825afab2f-tls-certs\") pod \"0ab619b4-ab20-4655-8def-7ed825afab2f\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") "
Oct 30 13:19:53.998728 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.998706    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-model-cache" pod "0ab619b4-ab20-4655-8def-7ed825afab2f" (UID: "0ab619b4-ab20-4655-8def-7ed825afab2f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:53.998864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.998846    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-dshm\") pod \"0ab619b4-ab20-4655-8def-7ed825afab2f\" (UID: \"0ab619b4-ab20-4655-8def-7ed825afab2f\") "
Oct 30 13:19:53.999005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.998968    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-tmp-dir" pod "0ab619b4-ab20-4655-8def-7ed825afab2f" (UID: "0ab619b4-ab20-4655-8def-7ed825afab2f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:53.999005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.998986    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-home" pod "0ab619b4-ab20-4655-8def-7ed825afab2f" (UID: "0ab619b4-ab20-4655-8def-7ed825afab2f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:53.999410 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.999392    5781 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tmp-dir\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:53.999495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.999414    5781 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-tls-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:53.999495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.999426    5781 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-model-cache\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:53.999495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.999438    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4njjc\" (UniqueName: \"kubernetes.io/projected/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-kube-api-access-4njjc\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:53.999495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.999450    5781 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-dshm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:53.999495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.999463    5781 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-model-cache\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:53.999495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.999474    5781 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-tmp-dir\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:53.999495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.999485    5781 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99-home\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:53.999495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:53.999496    5781 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-home\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.000815 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.000793    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-dshm" pod "0ab619b4-ab20-4655-8def-7ed825afab2f" (UID: "0ab619b4-ab20-4655-8def-7ed825afab2f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:19:54.001244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.001220    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ab619b4-ab20-4655-8def-7ed825afab2f-tls-certs" pod "0ab619b4-ab20-4655-8def-7ed825afab2f" (UID: "0ab619b4-ab20-4655-8def-7ed825afab2f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:19:54.001244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.001233    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ab619b4-ab20-4655-8def-7ed825afab2f-kube-api-access-gqdqn" pod "0ab619b4-ab20-4655-8def-7ed825afab2f" (UID: "0ab619b4-ab20-4655-8def-7ed825afab2f"). InnerVolumeSpecName "kube-api-access-gqdqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:54.101111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.101072    5781 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ab619b4-ab20-4655-8def-7ed825afab2f-dshm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.101111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.101108    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqdqn\" (UniqueName: \"kubernetes.io/projected/0ab619b4-ab20-4655-8def-7ed825afab2f-kube-api-access-gqdqn\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.101111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.101119    5781 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ab619b4-ab20-4655-8def-7ed825afab2f-tls-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.231059 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.231013    5781 generic.go:356] "Generic (PLEG): container finished" podID="7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" containerID="3d451cb5cca7852ed1daa13df4001962c1cc8a3a575f72f36b1590191fc58864" exitCode=0
Oct 30 13:19:54.231264 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.231197    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r" event={"ID":"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f","Type":"ContainerDied","Data":"3d451cb5cca7852ed1daa13df4001962c1cc8a3a575f72f36b1590191fc58864"}
Oct 30 13:19:54.233068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.233042    5781 generic.go:356] "Generic (PLEG): container finished" podID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerID="716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7" exitCode=0
Oct 30 13:19:54.233197 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.233115    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"
Oct 30 13:19:54.233197 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.233132    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" event={"ID":"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99","Type":"ContainerDied","Data":"716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7"}
Oct 30 13:19:54.233197 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.233180    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg" event={"ID":"520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99","Type":"ContainerDied","Data":"998c680aaa6f15610955cf235241c31a97ff0b0c470674177fbf326a16a92736"}
Oct 30 13:19:54.233386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.233203    5781 scope.go:122] "RemoveContainer" containerID="716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7"
Oct 30 13:19:54.234827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.234748    5781 generic.go:356] "Generic (PLEG): container finished" podID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerID="13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737" exitCode=0
Oct 30 13:19:54.235961 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.235941    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"
Oct 30 13:19:54.239960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.239935    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" event={"ID":"0ab619b4-ab20-4655-8def-7ed825afab2f","Type":"ContainerDied","Data":"13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737"}
Oct 30 13:19:54.240337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.239973    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg" event={"ID":"0ab619b4-ab20-4655-8def-7ed825afab2f","Type":"ContainerDied","Data":"ef929ede46d9c7b3dd32d02705a462c8e4efc12346f16336caafaf267fca803f"}
Oct 30 13:19:54.240416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.240333    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:19:54.251313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.248149    5781 scope.go:122] "RemoveContainer" containerID="716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7"
Oct 30 13:19:54.251313 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:54.248490    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7\": container with ID starting with 716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7 not found: ID does not exist" containerID="716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7"
Oct 30 13:19:54.251313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.248524    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7"} err="failed to get container status \"716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7\": rpc error: code = NotFound desc = could not find container \"716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7\": container with ID starting with 716ab489011095cd74d1e0877347f26b9c1c2c106b0ba4798c31f4ea220adcc7 not found: ID does not exist"
Oct 30 13:19:54.251313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.248548    5781 scope.go:122] "RemoveContainer" containerID="13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737"
Oct 30 13:19:54.260042 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.260023    5781 scope.go:122] "RemoveContainer" containerID="13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737"
Oct 30 13:19:54.260378 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:19:54.260338    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737\": container with ID starting with 13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737 not found: ID does not exist" containerID="13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737"
Oct 30 13:19:54.260470 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.260389    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737"} err="failed to get container status \"13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737\": rpc error: code = NotFound desc = could not find container \"13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737\": container with ID starting with 13989cbf9f49d2770e3247e23718b3a5f44eff3f145bc371e9415f10578ee737 not found: ID does not exist"
Oct 30 13:19:54.287864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.287828    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"]
Oct 30 13:19:54.292005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.291984    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/premium-w1-05942d-kserve-56c45f49bf-f6ttg"]
Oct 30 13:19:54.312953 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.312928    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:19:54.434140 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.434104    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"]
Oct 30 13:19:54.440615 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.440587    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-05942d/simulator-w1-05942d-kserve-dc9f8bdcb-64dxg"]
Oct 30 13:19:54.506869 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.506786    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-kube-api-access-hrbvw\" (UniqueName: \"kubernetes.io/projected/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-kube-api-access-hrbvw\") pod \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\" (UID: \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\") "
Oct 30 13:19:54.507032 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.507017    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-plugins-config-volume\") pod \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\" (UID: \"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f\") "
Oct 30 13:19:54.507408 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.507381    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-plugins-config-volume" pod "7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" (UID: "7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:19:54.509276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.509256    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-kube-api-access-hrbvw" pod "7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" (UID: "7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f"). InnerVolumeSpecName "kube-api-access-hrbvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:19:54.608758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.608719    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.608758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.608752    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrbvw\" (UniqueName: \"kubernetes.io/projected/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f-kube-api-access-hrbvw\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:19:54.843006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.842929    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" path="/var/lib/kubelet/pods/0ab619b4-ab20-4655-8def-7ed825afab2f/volumes"
Oct 30 13:19:54.843429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.843412    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" path="/var/lib/kubelet/pods/520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99/volumes"
Oct 30 13:19:54.877256 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:54.877232    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:55.245061 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:55.245030    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"
Oct 30 13:19:55.245272 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:55.245067    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r" event={"ID":"7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f","Type":"ContainerDied","Data":"1ba7829c6fc5e51d3f16013776f262f708df967c62c2888ed7b9c57aef12f981"}
Oct 30 13:19:55.245272 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:55.245122    5781 scope.go:122] "RemoveContainer" containerID="3d451cb5cca7852ed1daa13df4001962c1cc8a3a575f72f36b1590191fc58864"
Oct 30 13:19:55.300489 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:55.300451    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"]
Oct 30 13:19:55.316480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:55.316446    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-031737-7bc9bb8dc5-8h67r"]
Oct 30 13:19:56.730476 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:56.730434    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:56.842843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:56.842814    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" path="/var/lib/kubelet/pods/7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f/volumes"
Oct 30 13:19:58.339858 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:58.339814    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:19:59.718005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:59.717972    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"]
Oct 30 13:19:59.723435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:59.723398    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs" podUID="19cd30f1-725e-4ffa-855e-8a80e928ce86" containerName="maas-api" containerID="cri-o://1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f" gracePeriod=30
Oct 30 13:19:59.889852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:19:59.889820    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:00.058503 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.058173    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:20:00.084728 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.084695    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19cd30f1-725e-4ffa-855e-8a80e928ce86-kube-api-access-grwsv\" (UniqueName: \"kubernetes.io/projected/19cd30f1-725e-4ffa-855e-8a80e928ce86-kube-api-access-grwsv\") pod \"19cd30f1-725e-4ffa-855e-8a80e928ce86\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") "
Oct 30 13:20:00.084888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.084788    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-metrics-certs\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-metrics-certs\") pod \"19cd30f1-725e-4ffa-855e-8a80e928ce86\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") "
Oct 30 13:20:00.085252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.084979    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19cd30f1-725e-4ffa-855e-8a80e928ce86-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19cd30f1-725e-4ffa-855e-8a80e928ce86-openshift-service-ca\") pod \"19cd30f1-725e-4ffa-855e-8a80e928ce86\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") "
Oct 30 13:20:00.085252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.085099    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls\") pod \"19cd30f1-725e-4ffa-855e-8a80e928ce86\" (UID: \"19cd30f1-725e-4ffa-855e-8a80e928ce86\") "
Oct 30 13:20:00.085610 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.085573    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19cd30f1-725e-4ffa-855e-8a80e928ce86-openshift-service-ca" pod "19cd30f1-725e-4ffa-855e-8a80e928ce86" (UID: "19cd30f1-725e-4ffa-855e-8a80e928ce86"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:00.087082 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.087056    5781 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/19cd30f1-725e-4ffa-855e-8a80e928ce86-openshift-service-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:00.088057 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.088014    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-metrics-certs" pod "19cd30f1-725e-4ffa-855e-8a80e928ce86" (UID: "19cd30f1-725e-4ffa-855e-8a80e928ce86"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:00.088181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.088078    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19cd30f1-725e-4ffa-855e-8a80e928ce86-kube-api-access-grwsv" pod "19cd30f1-725e-4ffa-855e-8a80e928ce86" (UID: "19cd30f1-725e-4ffa-855e-8a80e928ce86"). InnerVolumeSpecName "kube-api-access-grwsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:00.088600 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.088568    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls" pod "19cd30f1-725e-4ffa-855e-8a80e928ce86" (UID: "19cd30f1-725e-4ffa-855e-8a80e928ce86"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:00.188180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.188139    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grwsv\" (UniqueName: \"kubernetes.io/projected/19cd30f1-725e-4ffa-855e-8a80e928ce86-kube-api-access-grwsv\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:00.188180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.188166    5781 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-metrics-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:00.188180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.188179    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/19cd30f1-725e-4ffa-855e-8a80e928ce86-maas-api-tls\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:00.287373 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.287264    5781 generic.go:356] "Generic (PLEG): container finished" podID="19cd30f1-725e-4ffa-855e-8a80e928ce86" containerID="1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f" exitCode=0
Oct 30 13:20:00.287373 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.287324    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"
Oct 30 13:20:00.287599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.287323    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs" event={"ID":"19cd30f1-725e-4ffa-855e-8a80e928ce86","Type":"ContainerDied","Data":"1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f"}
Oct 30 13:20:00.287599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.287478    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs" event={"ID":"19cd30f1-725e-4ffa-855e-8a80e928ce86","Type":"ContainerDied","Data":"1b5a06f79b746235833944787790643de54caa3ece9081fdcbb25146b3e8f7f5"}
Oct 30 13:20:00.287599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.287511    5781 scope.go:122] "RemoveContainer" containerID="1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f"
Oct 30 13:20:00.302924 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.302902    5781 scope.go:122] "RemoveContainer" containerID="1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f"
Oct 30 13:20:00.303225 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:00.303205    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f\": container with ID starting with 1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f not found: ID does not exist" containerID="1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f"
Oct 30 13:20:00.303365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.303232    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f"} err="failed to get container status \"1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f\": rpc error: code = NotFound desc = could not find container \"1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f\": container with ID starting with 1511d6076235a46ee4364c9c5de3a0cbc5317435d2213f2bf0b9cd62cf0ee74f not found: ID does not exist"
Oct 30 13:20:00.367863 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.367831    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"]
Oct 30 13:20:00.400050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.400025    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-05942d-65fdbfc9bf-lwqrs"]
Oct 30 13:20:00.406244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.406219    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:00.661699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.661617    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:00.849449 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.849412    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19cd30f1-725e-4ffa-855e-8a80e928ce86" path="/var/lib/kubelet/pods/19cd30f1-725e-4ffa-855e-8a80e928ce86/volumes"
Oct 30 13:20:00.849914 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:00.849677    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:01.059595 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:01.059158    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz" podUID="7c951714-8bf1-4df2-af09-2c33656636d6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:20:01.647721 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:01.634634    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:01.841697 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:01.841666    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.071692    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"]
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072684    5781 cpu_manager.go:414] "containerMap: removing container" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072708    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072739    5781 cpu_manager.go:414] "containerMap: removing container" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072747    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072775    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072783    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072811    5781 cpu_manager.go:414] "containerMap: removing container" podUID="e3e2b109-3318-4037-8d35-ad4aea11ce83" containerName="revoke-keys"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072818    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3e2b109-3318-4037-8d35-ad4aea11ce83" containerName="revoke-keys"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072833    5781 cpu_manager.go:414] "containerMap: removing container" podUID="7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" containerName="payload-pre-processing"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072844    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" containerName="payload-pre-processing"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072884    5781 cpu_manager.go:414] "containerMap: removing container" podUID="19cd30f1-725e-4ffa-855e-8a80e928ce86" containerName="maas-api"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.072891    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19cd30f1-725e-4ffa-855e-8a80e928ce86" containerName="maas-api"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.073034    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3e2b109-3318-4037-8d35-ad4aea11ce83" containerName="revoke-keys"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.073048    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a0f9b16-2f25-4a1a-b7a9-803fa6939b7f" containerName="payload-pre-processing"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.073067    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="19cd30f1-725e-4ffa-855e-8a80e928ce86" containerName="maas-api"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.073079    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ebb19b0-d8c8-4f17-aa13-78f210c82265" containerName="istio-proxy"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.073091    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="520ca5aa-6058-4d3b-8ab0-75e6fdf6fb99" containerName="main"
Oct 30 13:20:02.141351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.073103    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ab619b4-ab20-4655-8def-7ed825afab2f" containerName="main"
Oct 30 13:20:02.158328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.158084    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.212482 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:02.183587    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:20:02.368317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.323400    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.368317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.323450    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.368317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.323547    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.368317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.323578    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.368317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.323620    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd695c3-1426-4214-a757-704c120075cb-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.368317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.323680    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.368317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.323710    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7kw6\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-kube-api-access-r7kw6\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.368317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.323763    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd695c3-1426-4214-a757-704c120075cb-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.368317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.323871    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.698312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.680544    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 13:20:02.743347 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.740083    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.743347 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.740242    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.743347 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.740315    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.743347 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.740470    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.743347 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.740537    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.841728 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.841677    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:02.972367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.971771    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd695c3-1426-4214-a757-704c120075cb-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.972367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.972146    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.972367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.972207    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7kw6\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-kube-api-access-r7kw6\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:02.972367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:02.972316    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd695c3-1426-4214-a757-704c120075cb-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:03.303361 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.275613    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd695c3-1426-4214-a757-704c120075cb-istiod-ca-cert\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:03.536700 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.338642    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-envoy\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:03.562721 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.364351    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:03.562995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.364391    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7" podUID="ba75e294-c7ff-4681-b4a2-f1892bded953" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/ready\": context deadline exceeded"
Oct 30 13:20:03.564235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.368647    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-data\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:03.564417 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.497784    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:03.564624 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.541343    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-certs\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:03.595685 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.582445    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-credential-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:03.610657 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.591652    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-socket\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:03.802349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.798481    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:03.813523 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.804787    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-istio-token\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:03.820414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.820375    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:03.878139 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.860033    5781 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:03.920310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.918693    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": context deadline exceeded"
Oct 30 13:20:03.963458 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.963410    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:03.963807 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.963772    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/ready\": context deadline exceeded"
Oct 30 13:20:03.984308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:03.969862    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:04.100361 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.059952    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd695c3-1426-4214-a757-704c120075cb-istio-podinfo\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:04.175312 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:04.145808    5781 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.265s"
Oct 30 13:20:04.175312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.145902    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:20:04.175312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.145932    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"]
Oct 30 13:20:04.175312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.145997    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:20:04.175312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.156728    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7kw6\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-kube-api-access-r7kw6\") pod \"e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") " pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:04.294314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.276495    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:04.420454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.419382    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:20:04.420454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.419459    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:04.453319 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.443764    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:04.488779 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.488372    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:20:04.488779 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.488431    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:04.743312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.709365    5781 patch_prober.go:29] interesting pod/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:20:04.743312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.709432    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:04.825501 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.825454    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:04.940453 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.940418    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:04.983311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.952275    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:20:04.994314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.964383    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:05.002314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.964422    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:05.002818 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:04.999814    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:20:05.526318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:05.496947    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:05.826509 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:05.826465    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:05.996320 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:05.994195    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl" podUID="80c5e747-d78e-4007-bbae-5f3e9f47f303" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:20:05.996320 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:05.995325    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/ready\": context deadline exceeded"
Oct 30 13:20:06.004337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.004306    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:20:06.146659 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.146179    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:06.291005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.289933    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"]
Oct 30 13:20:06.317895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.317736    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:20:06.318165 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.318143    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:20:06.318328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.318308    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:20:06.318474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.318452    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:20:06.372655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.372621    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:20:06.443977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.443948    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:20:06.457015 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.456993    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl"
Oct 30 13:20:06.953438 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:06.953399    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:07.046760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:07.046719    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" event={"ID":"8dd695c3-1426-4214-a757-704c120075cb","Type":"ContainerStarted","Data":"da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e"}
Oct 30 13:20:07.046978 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:07.046800    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" event={"ID":"8dd695c3-1426-4214-a757-704c120075cb","Type":"ContainerStarted","Data":"a7da27d8c370ade327ef47bd18b891c836366c9a197b7719920ca3a2b3bd6da5"}
Oct 30 13:20:07.141687 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:07.141618    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" podStartSLOduration=6.141597197 podStartE2EDuration="6.141597197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:07.106758765 +0000 UTC m=+1344.403976308" watchObservedRunningTime="2026-10-30 13:20:07.141597197 +0000 UTC m=+1344.438814740"
Oct 30 13:20:07.349613 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:07.348345    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:07.349613 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:07.349529    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:20:07.349613 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:07.349576    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 13:20:08.334309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:08.325594    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:20:08.334309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:08.325653    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 13:20:08.697700 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:08.697661    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:09.337731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:09.337696    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:20:09.338214 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:09.337757    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 13:20:10.071029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.070994    5781 generic.go:356] "Generic (PLEG): container finished" podID="7c951714-8bf1-4df2-af09-2c33656636d6" containerID="2edd0f5d4bc95c1b64ea31e3bac23ec3f707ab2c55074287ac411c91134d8494" exitCode=0
Oct 30 13:20:10.071191 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.071065    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz" event={"ID":"7c951714-8bf1-4df2-af09-2c33656636d6","Type":"ContainerDied","Data":"2edd0f5d4bc95c1b64ea31e3bac23ec3f707ab2c55074287ac411c91134d8494"}
Oct 30 13:20:10.236311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.222737    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:20:10.298429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.298387    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:10.345617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.338661    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:20:10.345617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.338718    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 13:20:10.361382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.360617    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c951714-8bf1-4df2-af09-2c33656636d6-kube-api-access-dxl2w\" (UniqueName: \"kubernetes.io/projected/7c951714-8bf1-4df2-af09-2c33656636d6-kube-api-access-dxl2w\") pod \"7c951714-8bf1-4df2-af09-2c33656636d6\" (UID: \"7c951714-8bf1-4df2-af09-2c33656636d6\") "
Oct 30 13:20:10.361382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.360669    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c951714-8bf1-4df2-af09-2c33656636d6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c951714-8bf1-4df2-af09-2c33656636d6-plugins-config-volume\") pod \"7c951714-8bf1-4df2-af09-2c33656636d6\" (UID: \"7c951714-8bf1-4df2-af09-2c33656636d6\") "
Oct 30 13:20:10.361682 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.361499    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c951714-8bf1-4df2-af09-2c33656636d6-plugins-config-volume" pod "7c951714-8bf1-4df2-af09-2c33656636d6" (UID: "7c951714-8bf1-4df2-af09-2c33656636d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:10.370500 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.370461    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c951714-8bf1-4df2-af09-2c33656636d6-kube-api-access-dxl2w" pod "7c951714-8bf1-4df2-af09-2c33656636d6" (UID: "7c951714-8bf1-4df2-af09-2c33656636d6"). InnerVolumeSpecName "kube-api-access-dxl2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:10.462688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.462615    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxl2w\" (UniqueName: \"kubernetes.io/projected/7c951714-8bf1-4df2-af09-2c33656636d6-kube-api-access-dxl2w\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:10.462688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:10.462639    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7c951714-8bf1-4df2-af09-2c33656636d6-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:11.079949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.079913    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"
Oct 30 13:20:11.079949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.079940    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz" event={"ID":"7c951714-8bf1-4df2-af09-2c33656636d6","Type":"ContainerDied","Data":"1eef3844ae2a9919aa3016805a3b686d50a771dfdc42c16954ec988aae7a5ece"}
Oct 30 13:20:11.080175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.080030    5781 scope.go:122] "RemoveContainer" containerID="2edd0f5d4bc95c1b64ea31e3bac23ec3f707ab2c55074287ac411c91134d8494"
Oct 30 13:20:11.145278 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.145205    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"]
Oct 30 13:20:11.155970 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.155943    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-05942d-55585b4bd7-kxcgz"]
Oct 30 13:20:11.333320 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.324705    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:20:11.333320 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.324761    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 13:20:11.456308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.456092    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:11.691413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.690211    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"]
Oct 30 13:20:11.691872 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.691841    5781 cpu_manager.go:414] "containerMap: removing container" podUID="7c951714-8bf1-4df2-af09-2c33656636d6" containerName="payload-pre-processing"
Oct 30 13:20:11.691872 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.691871    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c951714-8bf1-4df2-af09-2c33656636d6" containerName="payload-pre-processing"
Oct 30 13:20:11.692266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.692243    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c951714-8bf1-4df2-af09-2c33656636d6" containerName="payload-pre-processing"
Oct 30 13:20:11.707697 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.707631    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.733872 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.733833    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"]
Oct 30 13:20:11.750765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.750730    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"]
Oct 30 13:20:11.786302 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.786263    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:20:11.786438 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.786334    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:20:11.795672 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.795650    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fb15dd07-dlhzd\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.795791 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.795767    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c47mj\" (UniqueName: \"kubernetes.io/projected/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-kube-api-access-c47mj\") pod \"maas-api-revoke-sub-fb15dd07-dlhzd\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.795857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.795837    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fb15dd07-dlhzd\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.896844 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.896813    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fb15dd07-dlhzd\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.897045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.896914    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c47mj\" (UniqueName: \"kubernetes.io/projected/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-kube-api-access-c47mj\") pod \"maas-api-revoke-sub-fb15dd07-dlhzd\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.897045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.896953    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fb15dd07-dlhzd\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.897567 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.897540    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fb15dd07-dlhzd\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.899246 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.899226    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fb15dd07-dlhzd\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.940603 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.940569    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c47mj\" (UniqueName: \"kubernetes.io/projected/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-kube-api-access-c47mj\") pod \"maas-api-revoke-sub-fb15dd07-dlhzd\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:11.956465 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:11.956430    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:12.063544 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:12.063463    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:12.162184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:12.162148    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:12.321255 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:12.321183    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:20:12.321255 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:12.321242    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.109:15021/healthz/ready\": dial tcp 10.134.0.109:15021: connect: connection refused"
Oct 30 13:20:12.432282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:12.432257    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"]
Oct 30 13:20:12.435043 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:20:12.435015    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb4f7f2_0b7d_4aec_a49f_0811a1b16021.slice/crio-91f647b38196d00a878ee11918ee0aa08cff4cae3b5d9b4dfdb4af52602844f2 WatchSource:0}: Error finding container 91f647b38196d00a878ee11918ee0aa08cff4cae3b5d9b4dfdb4af52602844f2: Status 404 returned error can't find the container with id 91f647b38196d00a878ee11918ee0aa08cff4cae3b5d9b4dfdb4af52602844f2
Oct 30 13:20:12.468751 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:12.468726    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:12.860204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:12.860164    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c951714-8bf1-4df2-af09-2c33656636d6" path="/var/lib/kubelet/pods/7c951714-8bf1-4df2-af09-2c33656636d6/volumes"
Oct 30 13:20:13.104045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:13.104004    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd" event={"ID":"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021","Type":"ContainerStarted","Data":"750f9a65079e1a1c6c0a59380000b9a4eb89f7560d6ebbf4959be544593fbe02"}
Oct 30 13:20:13.104251 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:13.104065    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd" event={"ID":"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021","Type":"ContainerStarted","Data":"91f647b38196d00a878ee11918ee0aa08cff4cae3b5d9b4dfdb4af52602844f2"}
Oct 30 13:20:13.167261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:13.167213    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd" podStartSLOduration=2.167199609 podStartE2EDuration="2.167199609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:13.166601065 +0000 UTC m=+1350.463818607" watchObservedRunningTime="2026-10-30 13:20:13.167199609 +0000 UTC m=+1350.464417151"
Oct 30 13:20:13.424218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:13.424187    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:13.424907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:13.424886    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:13.427179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:13.427158    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:20:13.990410 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:13.990373    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:14.113011 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:14.112973    5781 generic.go:356] "Generic (PLEG): container finished" podID="3cb4f7f2-0b7d-4aec-a49f-0811a1b16021" containerID="750f9a65079e1a1c6c0a59380000b9a4eb89f7560d6ebbf4959be544593fbe02" exitCode=0
Oct 30 13:20:14.113201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:14.113051    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd" event={"ID":"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021","Type":"ContainerDied","Data":"750f9a65079e1a1c6c0a59380000b9a4eb89f7560d6ebbf4959be544593fbe02"}
Oct 30 13:20:15.433885 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.433857    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:15.582219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.582182    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-kube-api-access-c47mj\" (UniqueName: \"kubernetes.io/projected/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-kube-api-access-c47mj\") pod \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") "
Oct 30 13:20:15.582449 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.582281    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-xks-ca\") pod \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") "
Oct 30 13:20:15.582537 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.582455    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-openshift-ca\") pod \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\" (UID: \"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021\") "
Oct 30 13:20:15.583115 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.583080    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-openshift-ca" pod "3cb4f7f2-0b7d-4aec-a49f-0811a1b16021" (UID: "3cb4f7f2-0b7d-4aec-a49f-0811a1b16021"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:15.583400 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.583375    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:15.585362 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.585334    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-xks-ca" pod "3cb4f7f2-0b7d-4aec-a49f-0811a1b16021" (UID: "3cb4f7f2-0b7d-4aec-a49f-0811a1b16021"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:15.585645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.585619    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-kube-api-access-c47mj" pod "3cb4f7f2-0b7d-4aec-a49f-0811a1b16021" (UID: "3cb4f7f2-0b7d-4aec-a49f-0811a1b16021"). InnerVolumeSpecName "kube-api-access-c47mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:15.684305 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.684210    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c47mj\" (UniqueName: \"kubernetes.io/projected/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-kube-api-access-c47mj\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:15.684305 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.684239    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:15.784096 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:15.784048    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:16.169362 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:16.168512    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"
Oct 30 13:20:16.169362 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:16.168562    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd" event={"ID":"3cb4f7f2-0b7d-4aec-a49f-0811a1b16021","Type":"ContainerDied","Data":"91f647b38196d00a878ee11918ee0aa08cff4cae3b5d9b4dfdb4af52602844f2"}
Oct 30 13:20:16.169362 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:16.168616    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91f647b38196d00a878ee11918ee0aa08cff4cae3b5d9b4dfdb4af52602844f2"
Oct 30 13:20:16.595855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:16.595737    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:16.744347 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:16.735353    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv" podUID="95db4022-c4ff-4a65-9142-d336e04cb839" containerName="payload-processing" containerID="cri-o://9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0" gracePeriod=25
Oct 30 13:20:17.011528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:17.011492    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv" podUID="95db4022-c4ff-4a65-9142-d336e04cb839" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:20:17.812590 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:17.812557    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:19.451006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:19.450972    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:19.619450 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:19.619414    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:20.141130 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.141027    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:20.404874 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.404800    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:20.561446 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.561414    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:20.751513 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.751477    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"]
Oct 30 13:20:20.751747 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.751719    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy" containerID="cri-o://1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0" gracePeriod=30
Oct 30 13:20:20.772233 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.772165    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"]
Oct 30 13:20:20.772544 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.772507    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy" containerID="cri-o://f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9" gracePeriod=30
Oct 30 13:20:20.778964 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.778936    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"]
Oct 30 13:20:20.779198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.779173    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy" containerID="cri-o://bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111" gracePeriod=30
Oct 30 13:20:20.788319 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.788276    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"]
Oct 30 13:20:20.788474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.788457    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy" containerID="cri-o://43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f" gracePeriod=30
Oct 30 13:20:20.904857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:20.904814    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:21.065896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:21.065820    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"]
Oct 30 13:20:21.072756 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:21.072724    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-94af0758-h54vx"]
Oct 30 13:20:21.176092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:21.176057    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:22.181624 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.181500    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:22.391604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.391569    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:22.446496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.446217    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"]
Oct 30 13:20:22.446664 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.446511    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy" containerID="cri-o://8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb" gracePeriod=30
Oct 30 13:20:22.453787 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.453764    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"]
Oct 30 13:20:22.453958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.453939    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy" containerID="cri-o://d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779" gracePeriod=30
Oct 30 13:20:22.468076 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.468053    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"]
Oct 30 13:20:22.468222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.468205    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" containerName="istio-proxy" containerID="cri-o://7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5" gracePeriod=30
Oct 30 13:20:22.478933 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.478912    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"]
Oct 30 13:20:22.479077 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.479061    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy" containerID="cri-o://a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273" gracePeriod=30
Oct 30 13:20:22.602427 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.602385    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:22.842328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:22.842225    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0dfc3095-8c23-49a8-bc62-530830f57e13" path="/var/lib/kubelet/pods/0dfc3095-8c23-49a8-bc62-530830f57e13/volumes"
Oct 30 13:20:23.124026 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:23.123942    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:24.902120 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:24.902079    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:25.910020 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:25.909973    5781 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96af49f4_900b_45ae_88a3_8126f203bf77.slice/crio-bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96af49f4_900b_45ae_88a3_8126f203bf77.slice/crio-conmon-bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 13:20:25.910441 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:25.910063    5781 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96af49f4_900b_45ae_88a3_8126f203bf77.slice/crio-bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96af49f4_900b_45ae_88a3_8126f203bf77.slice/crio-conmon-bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 13:20:26.097342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.097269    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:20:26.167984 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.167961    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:20:26.175446 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.175424    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:20:26.178844 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.178821    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:20:26.184449 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.184419    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-podinfo\") pod \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") "
Oct 30 13:20:26.184549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.184473    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-socket\") pod \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") "
Oct 30 13:20:26.184549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.184506    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-envoy\") pod \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") "
Oct 30 13:20:26.184658 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.184643    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-data\") pod \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") "
Oct 30 13:20:26.184713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.184675    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-token\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-token\") pod \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") "
Oct 30 13:20:26.184804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.184788    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-kube-api-access-zfhqm\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-kube-api-access-zfhqm\") pod \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") "
Oct 30 13:20:26.184871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.184810    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-socket" pod "596788d7-8cb7-43bd-a7ad-65ca6daf0a64" (UID: "596788d7-8cb7-43bd-a7ad-65ca6daf0a64"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.184871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.184845    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-certs\") pod \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") "
Oct 30 13:20:26.184976 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.184901    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-credential-socket\") pod \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") "
Oct 30 13:20:26.185187 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.185020    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-data" pod "596788d7-8cb7-43bd-a7ad-65ca6daf0a64" (UID: "596788d7-8cb7-43bd-a7ad-65ca6daf0a64"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.185187 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.185046    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istiod-ca-cert\") pod \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\" (UID: \"596788d7-8cb7-43bd-a7ad-65ca6daf0a64\") "
Oct 30 13:20:26.185275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.185226    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-certs" pod "596788d7-8cb7-43bd-a7ad-65ca6daf0a64" (UID: "596788d7-8cb7-43bd-a7ad-65ca6daf0a64"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.186707 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.185557    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istiod-ca-cert" pod "596788d7-8cb7-43bd-a7ad-65ca6daf0a64" (UID: "596788d7-8cb7-43bd-a7ad-65ca6daf0a64"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:26.186707 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.185794    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-credential-socket" pod "596788d7-8cb7-43bd-a7ad-65ca6daf0a64" (UID: "596788d7-8cb7-43bd-a7ad-65ca6daf0a64"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.186707 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.185819    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.186707 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.185838    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.186707 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.185850    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.186707 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.185864    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.188708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.188686    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-podinfo" pod "596788d7-8cb7-43bd-a7ad-65ca6daf0a64" (UID: "596788d7-8cb7-43bd-a7ad-65ca6daf0a64"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:26.188708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.188697    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-envoy" pod "596788d7-8cb7-43bd-a7ad-65ca6daf0a64" (UID: "596788d7-8cb7-43bd-a7ad-65ca6daf0a64"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.188816 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.188764    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-token" pod "596788d7-8cb7-43bd-a7ad-65ca6daf0a64" (UID: "596788d7-8cb7-43bd-a7ad-65ca6daf0a64"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.188816 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.188784    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-kube-api-access-zfhqm" pod "596788d7-8cb7-43bd-a7ad-65ca6daf0a64" (UID: "596788d7-8cb7-43bd-a7ad-65ca6daf0a64"). InnerVolumeSpecName "kube-api-access-zfhqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.261445 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.261408    5781 generic.go:356] "Generic (PLEG): container finished" podID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerID="43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f" exitCode=0
Oct 30 13:20:26.261845 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.261811    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" event={"ID":"356e2333-8a7f-40ca-b102-0f4812c5ceb3","Type":"ContainerDied","Data":"43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f"}
Oct 30 13:20:26.262000 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.261976    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf" event={"ID":"356e2333-8a7f-40ca-b102-0f4812c5ceb3","Type":"ContainerDied","Data":"324d23d7069cd28c51a2e6ef741079670e0e37025848c8c4498c57645589ff77"}
Oct 30 13:20:26.262082 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.262041    5781 scope.go:122] "RemoveContainer" containerID="43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f"
Oct 30 13:20:26.262194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.262178    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"
Oct 30 13:20:26.264071 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.264046    5781 generic.go:356] "Generic (PLEG): container finished" podID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerID="f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9" exitCode=0
Oct 30 13:20:26.264193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.264175    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" event={"ID":"596788d7-8cb7-43bd-a7ad-65ca6daf0a64","Type":"ContainerDied","Data":"f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9"}
Oct 30 13:20:26.264260 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.264212    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"
Oct 30 13:20:26.264260 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.264228    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7" event={"ID":"596788d7-8cb7-43bd-a7ad-65ca6daf0a64","Type":"ContainerDied","Data":"a2f2eaed04a07f4016a0c2b865923c5247da468d8e07660a59aecf7083c4fdb4"}
Oct 30 13:20:26.267013 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.266985    5781 generic.go:356] "Generic (PLEG): container finished" podID="96af49f4-900b-45ae-88a3-8126f203bf77" containerID="bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111" exitCode=0
Oct 30 13:20:26.267244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.267225    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"
Oct 30 13:20:26.267394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.267382    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" event={"ID":"96af49f4-900b-45ae-88a3-8126f203bf77","Type":"ContainerDied","Data":"bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111"}
Oct 30 13:20:26.267469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.267415    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq" event={"ID":"96af49f4-900b-45ae-88a3-8126f203bf77","Type":"ContainerDied","Data":"7f87cc8859f388450b0de4e4bc05986274969c113cd45e8e4fc2d13069ebce2c"}
Oct 30 13:20:26.270526 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.270503    5781 generic.go:356] "Generic (PLEG): container finished" podID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerID="1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0" exitCode=0
Oct 30 13:20:26.270732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.270542    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" event={"ID":"cd4e4e0f-5c48-4804-ad64-f9121adb82db","Type":"ContainerDied","Data":"1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0"}
Oct 30 13:20:26.270853 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.270838    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl" event={"ID":"cd4e4e0f-5c48-4804-ad64-f9121adb82db","Type":"ContainerDied","Data":"1f7bc0b7499bc1d18b77a75af886fb084b583ab82babe13fbfc629493174db08"}
Oct 30 13:20:26.270943 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.270573    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"
Oct 30 13:20:26.278342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.278323    5781 scope.go:122] "RemoveContainer" containerID="43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f"
Oct 30 13:20:26.278713 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:26.278689    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f\": container with ID starting with 43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f not found: ID does not exist" containerID="43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f"
Oct 30 13:20:26.278786 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.278726    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f"} err="failed to get container status \"43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f\": rpc error: code = NotFound desc = could not find container \"43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f\": container with ID starting with 43d96a81df6269c147e07c9903aec6bfd64f375a6d2b9034d5de72c3c45c7c4f not found: ID does not exist"
Oct 30 13:20:26.278786 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.278748    5781 scope.go:122] "RemoveContainer" containerID="f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9"
Oct 30 13:20:26.286966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.286914    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-certs\") pod \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") "
Oct 30 13:20:26.286966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.286961    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-certs\") pod \"96af49f4-900b-45ae-88a3-8126f203bf77\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") "
Oct 30 13:20:26.287120 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287035    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-envoy\") pod \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") "
Oct 30 13:20:26.287120 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287080    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-credential-socket\") pod \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") "
Oct 30 13:20:26.287222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287144    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96af49f4-900b-45ae-88a3-8126f203bf77-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96af49f4-900b-45ae-88a3-8126f203bf77-istiod-ca-cert\") pod \"96af49f4-900b-45ae-88a3-8126f203bf77\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") "
Oct 30 13:20:26.287222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287174    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-data\") pod \"96af49f4-900b-45ae-88a3-8126f203bf77\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") "
Oct 30 13:20:26.287222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287207    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-socket\") pod \"96af49f4-900b-45ae-88a3-8126f203bf77\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") "
Oct 30 13:20:26.287396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287243    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-data\") pod \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") "
Oct 30 13:20:26.287396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287274    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-envoy\") pod \"96af49f4-900b-45ae-88a3-8126f203bf77\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") "
Oct 30 13:20:26.287396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287323    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-credential-socket\") pod \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") "
Oct 30 13:20:26.287396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287357    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-istio-token\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-istio-token\") pod \"96af49f4-900b-45ae-88a3-8126f203bf77\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") "
Oct 30 13:20:26.287596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287390    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-certs" pod "356e2333-8a7f-40ca-b102-0f4812c5ceb3" (UID: "356e2333-8a7f-40ca-b102-0f4812c5ceb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.287596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287427    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/96af49f4-900b-45ae-88a3-8126f203bf77-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96af49f4-900b-45ae-88a3-8126f203bf77-istio-podinfo\") pod \"96af49f4-900b-45ae-88a3-8126f203bf77\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") "
Oct 30 13:20:26.287596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287477    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-socket\") pod \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") "
Oct 30 13:20:26.287596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287546    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istiod-ca-cert\") pod \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") "
Oct 30 13:20:26.287596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287577    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-data" pod "96af49f4-900b-45ae-88a3-8126f203bf77" (UID: "96af49f4-900b-45ae-88a3-8126f203bf77"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.287596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287585    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-kube-api-access-2nfkf\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-kube-api-access-2nfkf\") pod \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") "
Oct 30 13:20:26.287883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287659    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istiod-ca-cert\") pod \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") "
Oct 30 13:20:26.287883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287725    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-certs" pod "96af49f4-900b-45ae-88a3-8126f203bf77" (UID: "96af49f4-900b-45ae-88a3-8126f203bf77"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.287883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287738    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-socket\") pod \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") "
Oct 30 13:20:26.287883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287773    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-data\") pod \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") "
Oct 30 13:20:26.287883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287811    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-kube-api-access-965vb\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-kube-api-access-965vb\") pod \"96af49f4-900b-45ae-88a3-8126f203bf77\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") "
Oct 30 13:20:26.287883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287855    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-podinfo\") pod \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") "
Oct 30 13:20:26.288181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287899    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-credential-socket\") pod \"96af49f4-900b-45ae-88a3-8126f203bf77\" (UID: \"96af49f4-900b-45ae-88a3-8126f203bf77\") "
Oct 30 13:20:26.288181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287949    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-podinfo\") pod \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") "
Oct 30 13:20:26.288181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.287982    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-envoy\") pod \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") "
Oct 30 13:20:26.288181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.288012    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-certs\") pod \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") "
Oct 30 13:20:26.288181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.288041    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-token\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-token\") pod \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\" (UID: \"cd4e4e0f-5c48-4804-ad64-f9121adb82db\") "
Oct 30 13:20:26.288181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.288083    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-kube-api-access-ljzjn\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-kube-api-access-ljzjn\") pod \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") "
Oct 30 13:20:26.288181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.288131    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-token\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-token\") pod \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\" (UID: \"356e2333-8a7f-40ca-b102-0f4812c5ceb3\") "
Oct 30 13:20:26.288181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.288131    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-socket" pod "cd4e4e0f-5c48-4804-ad64-f9121adb82db" (UID: "cd4e4e0f-5c48-4804-ad64-f9121adb82db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.288576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.288419    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-credential-socket" pod "356e2333-8a7f-40ca-b102-0f4812c5ceb3" (UID: "356e2333-8a7f-40ca-b102-0f4812c5ceb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.288904 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.288881    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-credential-socket" pod "cd4e4e0f-5c48-4804-ad64-f9121adb82db" (UID: "cd4e4e0f-5c48-4804-ad64-f9121adb82db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.289188 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.289168    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-socket" pod "96af49f4-900b-45ae-88a3-8126f203bf77" (UID: "96af49f4-900b-45ae-88a3-8126f203bf77"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.289597 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.289578    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96af49f4-900b-45ae-88a3-8126f203bf77-istiod-ca-cert" pod "96af49f4-900b-45ae-88a3-8126f203bf77" (UID: "96af49f4-900b-45ae-88a3-8126f203bf77"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:26.290224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.289802    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-socket" pod "356e2333-8a7f-40ca-b102-0f4812c5ceb3" (UID: "356e2333-8a7f-40ca-b102-0f4812c5ceb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.290224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.289848    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istiod-ca-cert" pod "356e2333-8a7f-40ca-b102-0f4812c5ceb3" (UID: "356e2333-8a7f-40ca-b102-0f4812c5ceb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:26.290224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.289925    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-data" pod "356e2333-8a7f-40ca-b102-0f4812c5ceb3" (UID: "356e2333-8a7f-40ca-b102-0f4812c5ceb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.290224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.289949    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-data" pod "cd4e4e0f-5c48-4804-ad64-f9121adb82db" (UID: "cd4e4e0f-5c48-4804-ad64-f9121adb82db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.290534 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290320    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-credential-socket" pod "96af49f4-900b-45ae-88a3-8126f203bf77" (UID: "96af49f4-900b-45ae-88a3-8126f203bf77"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.290534 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290324    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istiod-ca-cert" pod "cd4e4e0f-5c48-4804-ad64-f9121adb82db" (UID: "cd4e4e0f-5c48-4804-ad64-f9121adb82db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:26.290534 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290512    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-certs" pod "cd4e4e0f-5c48-4804-ad64-f9121adb82db" (UID: "cd4e4e0f-5c48-4804-ad64-f9121adb82db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290841    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290869    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290886    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290902    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290917    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290931    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfhqm\" (UniqueName: \"kubernetes.io/projected/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-kube-api-access-zfhqm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290946    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290961    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290975    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.290989    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.291003    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291279 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.291016    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/596788d7-8cb7-43bd-a7ad-65ca6daf0a64-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.291987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.291410    5781 scope.go:122] "RemoveContainer" containerID="f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9"
Oct 30 13:20:26.291987 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:26.291767    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9\": container with ID starting with f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9 not found: ID does not exist" containerID="f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9"
Oct 30 13:20:26.291987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.291800    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9"} err="failed to get container status \"f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9\": rpc error: code = NotFound desc = could not find container \"f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9\": container with ID starting with f4daf5de00bd38704d3ceb6bdc2f41d0d519b32f651ca7f9772dcd88becb00d9 not found: ID does not exist"
Oct 30 13:20:26.291987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.291825    5781 scope.go:122] "RemoveContainer" containerID="bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111"
Oct 30 13:20:26.299951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.294251    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-kube-api-access-ljzjn" pod "356e2333-8a7f-40ca-b102-0f4812c5ceb3" (UID: "356e2333-8a7f-40ca-b102-0f4812c5ceb3"). InnerVolumeSpecName "kube-api-access-ljzjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.299951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.294340    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-envoy" pod "cd4e4e0f-5c48-4804-ad64-f9121adb82db" (UID: "cd4e4e0f-5c48-4804-ad64-f9121adb82db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.299951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.294340    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-token" pod "356e2333-8a7f-40ca-b102-0f4812c5ceb3" (UID: "356e2333-8a7f-40ca-b102-0f4812c5ceb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.299951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.294444    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-token" pod "cd4e4e0f-5c48-4804-ad64-f9121adb82db" (UID: "cd4e4e0f-5c48-4804-ad64-f9121adb82db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.299951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.294844    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-kube-api-access-965vb" pod "96af49f4-900b-45ae-88a3-8126f203bf77" (UID: "96af49f4-900b-45ae-88a3-8126f203bf77"). InnerVolumeSpecName "kube-api-access-965vb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.299951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.295864    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-envoy" pod "96af49f4-900b-45ae-88a3-8126f203bf77" (UID: "96af49f4-900b-45ae-88a3-8126f203bf77"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.299951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.296603    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-envoy" pod "356e2333-8a7f-40ca-b102-0f4812c5ceb3" (UID: "356e2333-8a7f-40ca-b102-0f4812c5ceb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:26.299951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.296755    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-podinfo" pod "cd4e4e0f-5c48-4804-ad64-f9121adb82db" (UID: "cd4e4e0f-5c48-4804-ad64-f9121adb82db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:26.299951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.299651    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-istio-token" pod "96af49f4-900b-45ae-88a3-8126f203bf77" (UID: "96af49f4-900b-45ae-88a3-8126f203bf77"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.304752 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.304727    5781 scope.go:122] "RemoveContainer" containerID="bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111"
Oct 30 13:20:26.305282 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:26.305135    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111\": container with ID starting with bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111 not found: ID does not exist" containerID="bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111"
Oct 30 13:20:26.305282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.305211    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111"} err="failed to get container status \"bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111\": rpc error: code = NotFound desc = could not find container \"bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111\": container with ID starting with bffbd7f3221289daf7005a4b319bacb1f70083eadbd4368f9c7d489c49c54111 not found: ID does not exist"
Oct 30 13:20:26.305282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.305236    5781 scope.go:122] "RemoveContainer" containerID="1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0"
Oct 30 13:20:26.313711 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.313685    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-kube-api-access-2nfkf" pod "cd4e4e0f-5c48-4804-ad64-f9121adb82db" (UID: "cd4e4e0f-5c48-4804-ad64-f9121adb82db"). InnerVolumeSpecName "kube-api-access-2nfkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:26.315436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.315401    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-podinfo" pod "356e2333-8a7f-40ca-b102-0f4812c5ceb3" (UID: "356e2333-8a7f-40ca-b102-0f4812c5ceb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:26.316364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.316343    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96af49f4-900b-45ae-88a3-8126f203bf77-istio-podinfo" pod "96af49f4-900b-45ae-88a3-8126f203bf77" (UID: "96af49f4-900b-45ae-88a3-8126f203bf77"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:26.325634 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.325563    5781 scope.go:122] "RemoveContainer" containerID="49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a"
Oct 30 13:20:26.340484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.339527    5781 scope.go:122] "RemoveContainer" containerID="1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0"
Oct 30 13:20:26.340484 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:26.339812    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0\": container with ID starting with 1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0 not found: ID does not exist" containerID="1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0"
Oct 30 13:20:26.340484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.339836    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0"} err="failed to get container status \"1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0\": rpc error: code = NotFound desc = could not find container \"1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0\": container with ID starting with 1a8865bd52f0b1463ef267bb3cef7c5de962adfacfd02573f82fd577fc820da0 not found: ID does not exist"
Oct 30 13:20:26.340484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.339856    5781 scope.go:122] "RemoveContainer" containerID="49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a"
Oct 30 13:20:26.340484 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:26.340073    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a\": container with ID starting with 49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a not found: ID does not exist" containerID="49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a"
Oct 30 13:20:26.340484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.340090    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a"} err="failed to get container status \"49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a\": rpc error: code = NotFound desc = could not find container \"49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a\": container with ID starting with 49b8024bca4696642e8c0ccac31a98f4c0a8d47dd8efb28a5fe234167e498f3a not found: ID does not exist"
Oct 30 13:20:26.343632 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.343611    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"]
Oct 30 13:20:26.351888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.351864    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-gr9p7"]
Oct 30 13:20:26.392890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392864    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392894    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392908    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96af49f4-900b-45ae-88a3-8126f203bf77-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392920    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392932    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nfkf\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-kube-api-access-2nfkf\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392941    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392950    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392957    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392966    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-965vb\" (UniqueName: \"kubernetes.io/projected/96af49f4-900b-45ae-88a3-8126f203bf77-kube-api-access-965vb\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392974    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392983    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96af49f4-900b-45ae-88a3-8126f203bf77-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392991    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.392998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.392999    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.393429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.393009    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.393429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.393016    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.393429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.393024    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljzjn\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-kube-api-access-ljzjn\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.393429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.393032    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.393429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.393039    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd4e4e0f-5c48-4804-ad64-f9121adb82db-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.393429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.393046    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96af49f4-900b-45ae-88a3-8126f203bf77-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.393429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.393053    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/356e2333-8a7f-40ca-b102-0f4812c5ceb3-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:26.615442 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.615410    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"]
Oct 30 13:20:26.621342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.621321    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-68shf"]
Oct 30 13:20:26.656783 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.656761    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"]
Oct 30 13:20:26.660459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.660432    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-tzphq"]
Oct 30 13:20:26.701487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.701437    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:26.701864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.701837    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"]
Oct 30 13:20:26.708885 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.708850    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-05942d-openshift-default-5bdbff6f6c-qg4wl"]
Oct 30 13:20:26.841784 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.841710    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" path="/var/lib/kubelet/pods/356e2333-8a7f-40ca-b102-0f4812c5ceb3/volumes"
Oct 30 13:20:26.842219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.842205    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" path="/var/lib/kubelet/pods/596788d7-8cb7-43bd-a7ad-65ca6daf0a64/volumes"
Oct 30 13:20:26.842690 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.842675    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" path="/var/lib/kubelet/pods/96af49f4-900b-45ae-88a3-8126f203bf77/volumes"
Oct 30 13:20:26.843377 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.843359    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" path="/var/lib/kubelet/pods/cd4e4e0f-5c48-4804-ad64-f9121adb82db/volumes"
Oct 30 13:20:26.886376 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:26.886344    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:27.007617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.007584    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv" podUID="95db4022-c4ff-4a65-9142-d336e04cb839" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:20:27.224359 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.224325    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:27.770349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.767742    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:20:27.860055 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.860029    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:20:27.865971 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.864860    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:20:27.871028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.871004    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:20:27.922603 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.922571    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-token\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-token\") pod \"2d5455b9-c745-4c33-9599-63fbb962dd0d\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") "
Oct 30 13:20:27.922603 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.922608    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-certs\") pod \"2d5455b9-c745-4c33-9599-63fbb962dd0d\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") "
Oct 30 13:20:27.922855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.922642    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-podinfo\") pod \"2d5455b9-c745-4c33-9599-63fbb962dd0d\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") "
Oct 30 13:20:27.922855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.922686    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d5455b9-c745-4c33-9599-63fbb962dd0d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d5455b9-c745-4c33-9599-63fbb962dd0d-istiod-ca-cert\") pod \"2d5455b9-c745-4c33-9599-63fbb962dd0d\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") "
Oct 30 13:20:27.922855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.922730    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-credential-socket\") pod \"2d5455b9-c745-4c33-9599-63fbb962dd0d\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") "
Oct 30 13:20:27.923068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.922857    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-kube-api-access-548rw\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-kube-api-access-548rw\") pod \"2d5455b9-c745-4c33-9599-63fbb962dd0d\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") "
Oct 30 13:20:27.923068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.922943    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-socket\") pod \"2d5455b9-c745-4c33-9599-63fbb962dd0d\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") "
Oct 30 13:20:27.923068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.922970    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-envoy\") pod \"2d5455b9-c745-4c33-9599-63fbb962dd0d\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") "
Oct 30 13:20:27.923068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.922977    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-certs" pod "2d5455b9-c745-4c33-9599-63fbb962dd0d" (UID: "2d5455b9-c745-4c33-9599-63fbb962dd0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.923275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.923077    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-data\") pod \"2d5455b9-c745-4c33-9599-63fbb962dd0d\" (UID: \"2d5455b9-c745-4c33-9599-63fbb962dd0d\") "
Oct 30 13:20:27.923275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.923104    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-credential-socket" pod "2d5455b9-c745-4c33-9599-63fbb962dd0d" (UID: "2d5455b9-c745-4c33-9599-63fbb962dd0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.923275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.923208    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-socket" pod "2d5455b9-c745-4c33-9599-63fbb962dd0d" (UID: "2d5455b9-c745-4c33-9599-63fbb962dd0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.923275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.923220    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d5455b9-c745-4c33-9599-63fbb962dd0d-istiod-ca-cert" pod "2d5455b9-c745-4c33-9599-63fbb962dd0d" (UID: "2d5455b9-c745-4c33-9599-63fbb962dd0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:27.923484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.923341    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-data" pod "2d5455b9-c745-4c33-9599-63fbb962dd0d" (UID: "2d5455b9-c745-4c33-9599-63fbb962dd0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:27.924524 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.924407    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.924524 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.924431    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.924524 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.924445    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.924524 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.924457    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d5455b9-c745-4c33-9599-63fbb962dd0d-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.924524 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.924469    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:27.925355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.925332    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-podinfo" pod "2d5455b9-c745-4c33-9599-63fbb962dd0d" (UID: "2d5455b9-c745-4c33-9599-63fbb962dd0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:27.925465 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.925445    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-token" pod "2d5455b9-c745-4c33-9599-63fbb962dd0d" (UID: "2d5455b9-c745-4c33-9599-63fbb962dd0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:27.925519 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.925466    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-kube-api-access-548rw" pod "2d5455b9-c745-4c33-9599-63fbb962dd0d" (UID: "2d5455b9-c745-4c33-9599-63fbb962dd0d"). InnerVolumeSpecName "kube-api-access-548rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:27.925847 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:27.925815    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-envoy" pod "2d5455b9-c745-4c33-9599-63fbb962dd0d" (UID: "2d5455b9-c745-4c33-9599-63fbb962dd0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.025976 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.025882    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-data\") pod \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") "
Oct 30 13:20:28.025976 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.025930    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-socket\") pod \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") "
Oct 30 13:20:28.025976 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.025963    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-token\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-token\") pod \"519f908a-bf41-4285-8ac4-1abe0a9d893e\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.025989    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-istio-token\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-istio-token\") pod \"c8211606-b218-40b9-aea3-c6d96034ba89\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026048    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-data\") pod \"519f908a-bf41-4285-8ac4-1abe0a9d893e\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026077    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-credential-socket\") pod \"519f908a-bf41-4285-8ac4-1abe0a9d893e\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026113    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-podinfo\") pod \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026144    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-envoy\") pod \"519f908a-bf41-4285-8ac4-1abe0a9d893e\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026189    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istiod-ca-cert\") pod \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026226    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-kube-api-access-hqv9p\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-kube-api-access-hqv9p\") pod \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026269    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-podinfo\") pod \"519f908a-bf41-4285-8ac4-1abe0a9d893e\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026277    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-data" pod "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" (UID: "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026348    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-token\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-token\") pod \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026381    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-data\") pod \"c8211606-b218-40b9-aea3-c6d96034ba89\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026425    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-envoy\") pod \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026473    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/519f908a-bf41-4285-8ac4-1abe0a9d893e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519f908a-bf41-4285-8ac4-1abe0a9d893e-istiod-ca-cert\") pod \"519f908a-bf41-4285-8ac4-1abe0a9d893e\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026479    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-credential-socket" pod "519f908a-bf41-4285-8ac4-1abe0a9d893e" (UID: "519f908a-bf41-4285-8ac4-1abe0a9d893e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026521    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-envoy\") pod \"c8211606-b218-40b9-aea3-c6d96034ba89\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") "
Oct 30 13:20:28.026553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026559    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8211606-b218-40b9-aea3-c6d96034ba89-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8211606-b218-40b9-aea3-c6d96034ba89-istiod-ca-cert\") pod \"c8211606-b218-40b9-aea3-c6d96034ba89\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026630    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-kube-api-access-87phn\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-kube-api-access-87phn\") pod \"519f908a-bf41-4285-8ac4-1abe0a9d893e\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026667    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-data" pod "519f908a-bf41-4285-8ac4-1abe0a9d893e" (UID: "519f908a-bf41-4285-8ac4-1abe0a9d893e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026675    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-certs\") pod \"519f908a-bf41-4285-8ac4-1abe0a9d893e\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026764    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-socket\") pod \"c8211606-b218-40b9-aea3-c6d96034ba89\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026775    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-socket" pod "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" (UID: "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026838    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-socket\") pod \"519f908a-bf41-4285-8ac4-1abe0a9d893e\" (UID: \"519f908a-bf41-4285-8ac4-1abe0a9d893e\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026881    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-credential-socket\") pod \"c8211606-b218-40b9-aea3-c6d96034ba89\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026949    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c8211606-b218-40b9-aea3-c6d96034ba89-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8211606-b218-40b9-aea3-c6d96034ba89-istio-podinfo\") pod \"c8211606-b218-40b9-aea3-c6d96034ba89\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.026989    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-credential-socket\") pod \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027021    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-certs\") pod \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\" (UID: \"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027050    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-kube-api-access-h6bh9\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-kube-api-access-h6bh9\") pod \"c8211606-b218-40b9-aea3-c6d96034ba89\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") "
Oct 30 13:20:28.027533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027080    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-certs\") pod \"c8211606-b218-40b9-aea3-c6d96034ba89\" (UID: \"c8211606-b218-40b9-aea3-c6d96034ba89\") "
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027792    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-548rw\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-kube-api-access-548rw\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027814    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027829    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027845    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027857    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027872    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d5455b9-c745-4c33-9599-63fbb962dd0d-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027885    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.027899    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.028086    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-certs" pod "c8211606-b218-40b9-aea3-c6d96034ba89" (UID: "c8211606-b218-40b9-aea3-c6d96034ba89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.028568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.028398    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8211606-b218-40b9-aea3-c6d96034ba89-istiod-ca-cert" pod "c8211606-b218-40b9-aea3-c6d96034ba89" (UID: "c8211606-b218-40b9-aea3-c6d96034ba89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:28.029200 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.028845    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-credential-socket" pod "c8211606-b218-40b9-aea3-c6d96034ba89" (UID: "c8211606-b218-40b9-aea3-c6d96034ba89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.029200 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.029121    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-certs" pod "519f908a-bf41-4285-8ac4-1abe0a9d893e" (UID: "519f908a-bf41-4285-8ac4-1abe0a9d893e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.029343 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.029318    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-socket" pod "c8211606-b218-40b9-aea3-c6d96034ba89" (UID: "c8211606-b218-40b9-aea3-c6d96034ba89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.029533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.029513    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-socket" pod "519f908a-bf41-4285-8ac4-1abe0a9d893e" (UID: "519f908a-bf41-4285-8ac4-1abe0a9d893e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.029732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.029708    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-token" pod "519f908a-bf41-4285-8ac4-1abe0a9d893e" (UID: "519f908a-bf41-4285-8ac4-1abe0a9d893e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:28.029806 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.029752    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-istio-token" pod "c8211606-b218-40b9-aea3-c6d96034ba89" (UID: "c8211606-b218-40b9-aea3-c6d96034ba89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:28.030597 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.030011    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istiod-ca-cert" pod "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" (UID: "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:28.030597 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.030310    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-data" pod "c8211606-b218-40b9-aea3-c6d96034ba89" (UID: "c8211606-b218-40b9-aea3-c6d96034ba89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.031022 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.030807    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-kube-api-access-87phn" pod "519f908a-bf41-4285-8ac4-1abe0a9d893e" (UID: "519f908a-bf41-4285-8ac4-1abe0a9d893e"). InnerVolumeSpecName "kube-api-access-87phn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:28.031022 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.030825    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-credential-socket" pod "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" (UID: "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.031171 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.031050    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-certs" pod "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" (UID: "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.031171 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.031093    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-podinfo" pod "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" (UID: "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:28.031831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.031806    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-envoy" pod "519f908a-bf41-4285-8ac4-1abe0a9d893e" (UID: "519f908a-bf41-4285-8ac4-1abe0a9d893e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.031927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.031910    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-kube-api-access-hqv9p" pod "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" (UID: "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b"). InnerVolumeSpecName "kube-api-access-hqv9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:28.032136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.032110    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-podinfo" pod "519f908a-bf41-4285-8ac4-1abe0a9d893e" (UID: "519f908a-bf41-4285-8ac4-1abe0a9d893e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:28.032220 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.032149    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/519f908a-bf41-4285-8ac4-1abe0a9d893e-istiod-ca-cert" pod "519f908a-bf41-4285-8ac4-1abe0a9d893e" (UID: "519f908a-bf41-4285-8ac4-1abe0a9d893e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:28.033642 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.033617    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-envoy" pod "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" (UID: "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.033755 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.033672    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-kube-api-access-h6bh9" pod "c8211606-b218-40b9-aea3-c6d96034ba89" (UID: "c8211606-b218-40b9-aea3-c6d96034ba89"). InnerVolumeSpecName "kube-api-access-h6bh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:28.033814 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.033783    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-token" pod "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" (UID: "d7e237b9-8fb9-4fa9-9986-bd2beae17e9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:28.033959 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.033937    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-envoy" pod "c8211606-b218-40b9-aea3-c6d96034ba89" (UID: "c8211606-b218-40b9-aea3-c6d96034ba89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:28.034951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.034925    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8211606-b218-40b9-aea3-c6d96034ba89-istio-podinfo" pod "c8211606-b218-40b9-aea3-c6d96034ba89" (UID: "c8211606-b218-40b9-aea3-c6d96034ba89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:28.129553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129521    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129548    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8211606-b218-40b9-aea3-c6d96034ba89-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129557    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129565    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129575    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6bh9\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-kube-api-access-h6bh9\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129584    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129592    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129599    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8211606-b218-40b9-aea3-c6d96034ba89-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129606    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129613    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129621    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129628    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqv9p\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-kube-api-access-hqv9p\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129635    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/519f908a-bf41-4285-8ac4-1abe0a9d893e-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129643    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129650    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129657    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129664    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/519f908a-bf41-4285-8ac4-1abe0a9d893e-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129670    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129677    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8211606-b218-40b9-aea3-c6d96034ba89-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129684    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87phn\" (UniqueName: \"kubernetes.io/projected/519f908a-bf41-4285-8ac4-1abe0a9d893e-kube-api-access-87phn\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129691    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129700    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8211606-b218-40b9-aea3-c6d96034ba89-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.129778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.129708    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/519f908a-bf41-4285-8ac4-1abe0a9d893e-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:28.274708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.274664    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:28.293024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.292957    5781 generic.go:356] "Generic (PLEG): container finished" podID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerID="a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273" exitCode=0
Oct 30 13:20:28.293024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.293007    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"
Oct 30 13:20:28.293211 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.293039    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" event={"ID":"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b","Type":"ContainerDied","Data":"a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273"}
Oct 30 13:20:28.293211 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.293090    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8" event={"ID":"d7e237b9-8fb9-4fa9-9986-bd2beae17e9b","Type":"ContainerDied","Data":"055b03d33edcd84490f1b5c3230f1d83776effc264587c588ff71fc46183940a"}
Oct 30 13:20:28.293211 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.293110    5781 scope.go:122] "RemoveContainer" containerID="a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273"
Oct 30 13:20:28.294912 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.294889    5781 generic.go:356] "Generic (PLEG): container finished" podID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerID="d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779" exitCode=0
Oct 30 13:20:28.295047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.294934    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"
Oct 30 13:20:28.295047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.294966    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" event={"ID":"519f908a-bf41-4285-8ac4-1abe0a9d893e","Type":"ContainerDied","Data":"d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779"}
Oct 30 13:20:28.295047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.295010    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d" event={"ID":"519f908a-bf41-4285-8ac4-1abe0a9d893e","Type":"ContainerDied","Data":"32d725537e5c4d01d8dcc2d7b853a43695f2fe0d6a1969a8eac70e0069731443"}
Oct 30 13:20:28.296618 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.296592    5781 generic.go:356] "Generic (PLEG): container finished" podID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerID="8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb" exitCode=0
Oct 30 13:20:28.296719 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.296631    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"
Oct 30 13:20:28.296788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.296719    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" event={"ID":"2d5455b9-c745-4c33-9599-63fbb962dd0d","Type":"ContainerDied","Data":"8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb"}
Oct 30 13:20:28.296788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.296757    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc" event={"ID":"2d5455b9-c745-4c33-9599-63fbb962dd0d","Type":"ContainerDied","Data":"f8a24fa1089c3346e95479e97ed72245ee634e2ce9081545cc502fd098aea4ca"}
Oct 30 13:20:28.298526 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.298501    5781 generic.go:356] "Generic (PLEG): container finished" podID="c8211606-b218-40b9-aea3-c6d96034ba89" containerID="7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5" exitCode=0
Oct 30 13:20:28.298628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.298548    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"
Oct 30 13:20:28.298699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.298548    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" event={"ID":"c8211606-b218-40b9-aea3-c6d96034ba89","Type":"ContainerDied","Data":"7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5"}
Oct 30 13:20:28.298699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.298681    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9" event={"ID":"c8211606-b218-40b9-aea3-c6d96034ba89","Type":"ContainerDied","Data":"5a50e45c487071f6a3eda649525dc974a7f491d43d5e25fca7015e840f017470"}
Oct 30 13:20:28.308467 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.308447    5781 scope.go:122] "RemoveContainer" containerID="a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273"
Oct 30 13:20:28.310246 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:28.310219    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273\": container with ID starting with a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273 not found: ID does not exist" containerID="a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273"
Oct 30 13:20:28.310424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.310257    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273"} err="failed to get container status \"a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273\": rpc error: code = NotFound desc = could not find container \"a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273\": container with ID starting with a4c953e55fccf60e12f8a00fba5009a7039ad83ddc0aee28c1dd9f1b76448273 not found: ID does not exist"
Oct 30 13:20:28.310424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.310282    5781 scope.go:122] "RemoveContainer" containerID="d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779"
Oct 30 13:20:28.324807 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.324785    5781 scope.go:122] "RemoveContainer" containerID="70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf"
Oct 30 13:20:28.334516 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.334492    5781 scope.go:122] "RemoveContainer" containerID="d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779"
Oct 30 13:20:28.334861 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:28.334808    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779\": container with ID starting with d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779 not found: ID does not exist" containerID="d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779"
Oct 30 13:20:28.334861 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.334844    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779"} err="failed to get container status \"d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779\": rpc error: code = NotFound desc = could not find container \"d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779\": container with ID starting with d7a07decfb2fac207cfe5de26f857fb93e39cc657d97dfef552b30878bbf7779 not found: ID does not exist"
Oct 30 13:20:28.334996 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.334866    5781 scope.go:122] "RemoveContainer" containerID="70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf"
Oct 30 13:20:28.335160 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:28.335138    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf\": container with ID starting with 70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf not found: ID does not exist" containerID="70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf"
Oct 30 13:20:28.335242 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.335169    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf"} err="failed to get container status \"70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf\": rpc error: code = NotFound desc = could not find container \"70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf\": container with ID starting with 70136a7cfd4e01fc4bff398859ea9250ee10b37c4f21ccb7ccc62b40e7413caf not found: ID does not exist"
Oct 30 13:20:28.335242 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.335188    5781 scope.go:122] "RemoveContainer" containerID="8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb"
Oct 30 13:20:28.344956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.344936    5781 scope.go:122] "RemoveContainer" containerID="8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb"
Oct 30 13:20:28.345268 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:28.345249    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb\": container with ID starting with 8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb not found: ID does not exist" containerID="8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb"
Oct 30 13:20:28.345375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.345276    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb"} err="failed to get container status \"8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb\": rpc error: code = NotFound desc = could not find container \"8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb\": container with ID starting with 8c008e35d12f9f2ca40d971fc17cd07292218b54d190628f273dc8110df51dfb not found: ID does not exist"
Oct 30 13:20:28.345375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.345311    5781 scope.go:122] "RemoveContainer" containerID="7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5"
Oct 30 13:20:28.354951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.354927    5781 scope.go:122] "RemoveContainer" containerID="7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5"
Oct 30 13:20:28.355246 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:28.355218    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5\": container with ID starting with 7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5 not found: ID does not exist" containerID="7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5"
Oct 30 13:20:28.355370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.355252    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5"} err="failed to get container status \"7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5\": rpc error: code = NotFound desc = could not find container \"7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5\": container with ID starting with 7d6d6cdddd3c64263ba5fc576f32c420afa30a89208a4f7858dd93fac6ebb0e5 not found: ID does not exist"
Oct 30 13:20:28.395502 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.395464    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"]
Oct 30 13:20:28.409119 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.409085    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-gx2gc"]
Oct 30 13:20:28.443390 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.443359    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"]
Oct 30 13:20:28.448828 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.448799    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-wlzr9"]
Oct 30 13:20:28.450869 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.450838    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:28.480782 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.480750    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"]
Oct 30 13:20:28.484952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.484925    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-7hh7d"]
Oct 30 13:20:28.524228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.524197    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"]
Oct 30 13:20:28.529234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.529211    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-031737-openshift-default-64b8f46f47-rtgp8"]
Oct 30 13:20:28.842030 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.841993    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" path="/var/lib/kubelet/pods/2d5455b9-c745-4c33-9599-63fbb962dd0d/volumes"
Oct 30 13:20:28.842517 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.842500    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" path="/var/lib/kubelet/pods/519f908a-bf41-4285-8ac4-1abe0a9d893e/volumes"
Oct 30 13:20:28.843112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.843088    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" path="/var/lib/kubelet/pods/c8211606-b218-40b9-aea3-c6d96034ba89/volumes"
Oct 30 13:20:28.843916 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:28.843899    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" path="/var/lib/kubelet/pods/d7e237b9-8fb9-4fa9-9986-bd2beae17e9b/volumes"
Oct 30 13:20:30.274266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:30.274236    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:31.054336 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:31.054309    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"]
Oct 30 13:20:31.065784 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:31.065761    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4b4a013c-9wh6m"]
Oct 30 13:20:31.563394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:31.563366    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:31.996385 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:31.996353    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:20:32.084035 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:32.083933    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:32.878312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:32.877970    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f" path="/var/lib/kubelet/pods/d2ff6ac5-e9c1-47bf-82b8-6c818dd0294f/volumes"
Oct 30 13:20:34.063591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:34.063556    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:35.858057 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:35.858028    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:36.059430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:36.059393    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:37.034062 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.034004    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"]
Oct 30 13:20:37.042462 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.042437    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:20:37.044763 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.044743    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-92666b8c-9bgc6"]
Oct 30 13:20:37.087350 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.087325    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95db4022-c4ff-4a65-9142-d336e04cb839-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95db4022-c4ff-4a65-9142-d336e04cb839-plugins-config-volume\") pod \"95db4022-c4ff-4a65-9142-d336e04cb839\" (UID: \"95db4022-c4ff-4a65-9142-d336e04cb839\") "
Oct 30 13:20:37.087493 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.087481    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95db4022-c4ff-4a65-9142-d336e04cb839-kube-api-access-4q987\" (UniqueName: \"kubernetes.io/projected/95db4022-c4ff-4a65-9142-d336e04cb839-kube-api-access-4q987\") pod \"95db4022-c4ff-4a65-9142-d336e04cb839\" (UID: \"95db4022-c4ff-4a65-9142-d336e04cb839\") "
Oct 30 13:20:37.087710 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.087687    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95db4022-c4ff-4a65-9142-d336e04cb839-plugins-config-volume" pod "95db4022-c4ff-4a65-9142-d336e04cb839" (UID: "95db4022-c4ff-4a65-9142-d336e04cb839"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:37.087805 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.087791    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95db4022-c4ff-4a65-9142-d336e04cb839-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:37.089441 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.089417    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95db4022-c4ff-4a65-9142-d336e04cb839-kube-api-access-4q987" pod "95db4022-c4ff-4a65-9142-d336e04cb839" (UID: "95db4022-c4ff-4a65-9142-d336e04cb839"). InnerVolumeSpecName "kube-api-access-4q987". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:37.189562 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.189529    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q987\" (UniqueName: \"kubernetes.io/projected/95db4022-c4ff-4a65-9142-d336e04cb839-kube-api-access-4q987\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:37.237853 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.237818    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:37.298345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.298268    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"]
Oct 30 13:20:37.298567 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.298519    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy" containerID="cri-o://e91e47d20a28194eb635b81c5c2de9664cd6f63c5cbfb26c6f87a679c94d4b3d" gracePeriod=30
Oct 30 13:20:37.408936 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.408906    5781 generic.go:356] "Generic (PLEG): container finished" podID="95db4022-c4ff-4a65-9142-d336e04cb839" containerID="9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0" exitCode=0
Oct 30 13:20:37.409114 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.408947    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"
Oct 30 13:20:37.409114 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.408987    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv" event={"ID":"95db4022-c4ff-4a65-9142-d336e04cb839","Type":"ContainerDied","Data":"9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0"}
Oct 30 13:20:37.409114 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.409047    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv" event={"ID":"95db4022-c4ff-4a65-9142-d336e04cb839","Type":"ContainerDied","Data":"51cc1640cb9b95d028eca6ecdb13788d6973bb9c4ff6e40e4902782777b363e1"}
Oct 30 13:20:37.409114 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.409076    5781 scope.go:122] "RemoveContainer" containerID="9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0"
Oct 30 13:20:37.418995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.418982    5781 scope.go:122] "RemoveContainer" containerID="9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0"
Oct 30 13:20:37.419245 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:37.419224    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0\": container with ID starting with 9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0 not found: ID does not exist" containerID="9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0"
Oct 30 13:20:37.419396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.419252    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0"} err="failed to get container status \"9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0\": rpc error: code = NotFound desc = could not find container \"9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0\": container with ID starting with 9e8e5418622812d3ab888f780eae4862577d66fa09d1862150b9731ccc1b57a0 not found: ID does not exist"
Oct 30 13:20:37.460510 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.460491    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"]
Oct 30 13:20:37.464799 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.464778    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv"]
Oct 30 13:20:37.719260 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.719230    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:37.981578 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:37.981511    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:38.007234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:38.007196    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-d0a18c-dcc4ddc54-9bknv" podUID="95db4022-c4ff-4a65-9142-d336e04cb839" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.49:50052\" within 1s: context deadline exceeded"
Oct 30 13:20:38.007502 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:20:38.007262    5781 logging.go:55] [core] [Channel #758 SubChannel #759]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.49:50052", ServerName: "10.134.0.49:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.49:50052: operation was canceled"
Oct 30 13:20:38.517735 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:38.517705    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:38.841676 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:38.841607    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16400144-2b71-4456-827d-046c6575a757" path="/var/lib/kubelet/pods/16400144-2b71-4456-827d-046c6575a757/volumes"
Oct 30 13:20:38.842006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:38.841992    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95db4022-c4ff-4a65-9142-d336e04cb839" path="/var/lib/kubelet/pods/95db4022-c4ff-4a65-9142-d336e04cb839/volumes"
Oct 30 13:20:39.776890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:39.776851    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:40.759201 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:40.759167    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:41.698851 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:41.698808    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:42.456521 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.456486    5781 generic.go:356] "Generic (PLEG): container finished" podID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerID="e91e47d20a28194eb635b81c5c2de9664cd6f63c5cbfb26c6f87a679c94d4b3d" exitCode=0
Oct 30 13:20:42.456701 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.456563    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" event={"ID":"9e8a198f-e34c-4216-bf22-ed9cf03d91f4","Type":"ContainerDied","Data":"e91e47d20a28194eb635b81c5c2de9664cd6f63c5cbfb26c6f87a679c94d4b3d"}
Oct 30 13:20:42.561838 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.561784    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:20:42.694265 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694233    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-credential-socket\") pod \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") "
Oct 30 13:20:42.694454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694331    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-token\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-token\") pod \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") "
Oct 30 13:20:42.694454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694378    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-kube-api-access-ldq7n\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-kube-api-access-ldq7n\") pod \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") "
Oct 30 13:20:42.694454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694437    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istiod-ca-cert\") pod \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") "
Oct 30 13:20:42.694628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694498    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-podinfo\") pod \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") "
Oct 30 13:20:42.694628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694530    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-credential-socket" pod "9e8a198f-e34c-4216-bf22-ed9cf03d91f4" (UID: "9e8a198f-e34c-4216-bf22-ed9cf03d91f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:42.694628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694585    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-envoy\") pod \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") "
Oct 30 13:20:42.694784 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694659    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-data\") pod \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") "
Oct 30 13:20:42.694784 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694731    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-certs\") pod \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") "
Oct 30 13:20:42.694907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694807    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-socket\") pod \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\" (UID: \"9e8a198f-e34c-4216-bf22-ed9cf03d91f4\") "
Oct 30 13:20:42.694907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694803    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istiod-ca-cert" pod "9e8a198f-e34c-4216-bf22-ed9cf03d91f4" (UID: "9e8a198f-e34c-4216-bf22-ed9cf03d91f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:42.695006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.694948    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-data" pod "9e8a198f-e34c-4216-bf22-ed9cf03d91f4" (UID: "9e8a198f-e34c-4216-bf22-ed9cf03d91f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:42.695192 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.695148    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-certs" pod "9e8a198f-e34c-4216-bf22-ed9cf03d91f4" (UID: "9e8a198f-e34c-4216-bf22-ed9cf03d91f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:42.695359 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.695206    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-socket" pod "9e8a198f-e34c-4216-bf22-ed9cf03d91f4" (UID: "9e8a198f-e34c-4216-bf22-ed9cf03d91f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:42.695482 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.695388    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:42.695482 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.695411    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:42.695482 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.695426    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:42.696762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.696723    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-token" pod "9e8a198f-e34c-4216-bf22-ed9cf03d91f4" (UID: "9e8a198f-e34c-4216-bf22-ed9cf03d91f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:42.696855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.696786    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-podinfo" pod "9e8a198f-e34c-4216-bf22-ed9cf03d91f4" (UID: "9e8a198f-e34c-4216-bf22-ed9cf03d91f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:42.696855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.696793    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-kube-api-access-ldq7n" pod "9e8a198f-e34c-4216-bf22-ed9cf03d91f4" (UID: "9e8a198f-e34c-4216-bf22-ed9cf03d91f4"). InnerVolumeSpecName "kube-api-access-ldq7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:42.696855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.696806    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-envoy" pod "9e8a198f-e34c-4216-bf22-ed9cf03d91f4" (UID: "9e8a198f-e34c-4216-bf22-ed9cf03d91f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:42.797021 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.796994    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:42.797021 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.797021    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:42.797021 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.797030    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:42.797480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.797039    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldq7n\" (UniqueName: \"kubernetes.io/projected/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-kube-api-access-ldq7n\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:42.797480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.797049    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:42.797480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:42.797056    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e8a198f-e34c-4216-bf22-ed9cf03d91f4-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:43.412504 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:43.412468    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:43.466434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:43.466403    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"
Oct 30 13:20:43.466563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:43.466444    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h" event={"ID":"9e8a198f-e34c-4216-bf22-ed9cf03d91f4","Type":"ContainerDied","Data":"76ffe8613da36cb8cd4cc9d3fa8654606ae33574ac7543e93119645e32a5ae41"}
Oct 30 13:20:43.466563 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:43.466510    5781 scope.go:122] "RemoveContainer" containerID="e91e47d20a28194eb635b81c5c2de9664cd6f63c5cbfb26c6f87a679c94d4b3d"
Oct 30 13:20:43.556757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:43.556729    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"]
Oct 30 13:20:43.566958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:43.566937    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-85w8h"]
Oct 30 13:20:44.199407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:44.199370    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:44.424764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:44.424732    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:44.841304 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:44.841215    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" path="/var/lib/kubelet/pods/9e8a198f-e34c-4216-bf22-ed9cf03d91f4/volumes"
Oct 30 13:20:45.287584 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:45.287553    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:45.707231 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:45.707203    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"]
Oct 30 13:20:45.726996 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:45.726972    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"]
Oct 30 13:20:45.786430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:45.786401    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"]
Oct 30 13:20:45.873270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:45.873244    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:46.147052 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:46.146979    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:47.066677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:47.066642    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"]
Oct 30 13:20:47.073749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:47.073724    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-68091082-r5245"]
Oct 30 13:20:47.121026 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:47.120998    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:48.341887 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.341686    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:48.527264 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.527223    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:48.787645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.786837    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"]
Oct 30 13:20:48.787645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.787058    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy" containerID="cri-o://d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e" gracePeriod=30
Oct 30 13:20:48.799856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.799823    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"]
Oct 30 13:20:48.800081 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.800059    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy" containerID="cri-o://816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1" gracePeriod=30
Oct 30 13:20:48.843731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.843700    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c2b5c11-ccc4-4d04-b999-4c7ee1501e24" path="/var/lib/kubelet/pods/3c2b5c11-ccc4-4d04-b999-4c7ee1501e24/volumes"
Oct 30 13:20:48.844037 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.844022    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"]
Oct 30 13:20:48.844213 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.844195    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy" containerID="cri-o://1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1" gracePeriod=30
Oct 30 13:20:48.878131 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:48.878075    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:49.310010 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:49.309914    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:50.328896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:50.328839    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:51.335546 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:51.335503    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:51.572263 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:51.572233    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:51.762240 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:51.762205    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:52.069340 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.069233    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"]
Oct 30 13:20:52.080870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.080838    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-ab936e4a-vcs6x"]
Oct 30 13:20:52.841788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.841756    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0297f575-381a-4fed-ad28-6d257c127d64" path="/var/lib/kubelet/pods/0297f575-381a-4fed-ad28-6d257c127d64/volumes"
Oct 30 13:20:52.870167 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870144    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"]
Oct 30 13:20:52.870701 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870659    5781 cpu_manager.go:414] "containerMap: removing container" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy"
Oct 30 13:20:52.870758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870704    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy"
Oct 30 13:20:52.870758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870721    5781 cpu_manager.go:414] "containerMap: removing container" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy"
Oct 30 13:20:52.870758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870735    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy"
Oct 30 13:20:52.870758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870749    5781 cpu_manager.go:414] "containerMap: removing container" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy"
Oct 30 13:20:52.870758 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870757    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870771    5781 cpu_manager.go:414] "containerMap: removing container" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870778    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870788    5781 cpu_manager.go:414] "containerMap: removing container" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870793    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870802    5781 cpu_manager.go:414] "containerMap: removing container" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870806    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870826    5781 cpu_manager.go:414] "containerMap: removing container" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870833    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870849    5781 cpu_manager.go:414] "containerMap: removing container" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870856    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870866    5781 cpu_manager.go:414] "containerMap: removing container" podUID="3cb4f7f2-0b7d-4aec-a49f-0811a1b16021" containerName="revoke-keys"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870873    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3cb4f7f2-0b7d-4aec-a49f-0811a1b16021" containerName="revoke-keys"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870890    5781 cpu_manager.go:414] "containerMap: removing container" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870894    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870900    5781 cpu_manager.go:414] "containerMap: removing container" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870904    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870914    5781 cpu_manager.go:414] "containerMap: removing container" podUID="95db4022-c4ff-4a65-9142-d336e04cb839" containerName="payload-processing"
Oct 30 13:20:52.870935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.870918    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95db4022-c4ff-4a65-9142-d336e04cb839" containerName="payload-processing"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871086    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7e237b9-8fb9-4fa9-9986-bd2beae17e9b" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871102    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="96af49f4-900b-45ae-88a3-8126f203bf77" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871113    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="95db4022-c4ff-4a65-9142-d336e04cb839" containerName="payload-processing"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871144    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e8a198f-e34c-4216-bf22-ed9cf03d91f4" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871154    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871163    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="596788d7-8cb7-43bd-a7ad-65ca6daf0a64" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871173    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd4e4e0f-5c48-4804-ad64-f9121adb82db" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871180    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8211606-b218-40b9-aea3-c6d96034ba89" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871189    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871221    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d5455b9-c745-4c33-9599-63fbb962dd0d" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871231    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="356e2333-8a7f-40ca-b102-0f4812c5ceb3" containerName="istio-proxy"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871240    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="3cb4f7f2-0b7d-4aec-a49f-0811a1b16021" containerName="revoke-keys"
Oct 30 13:20:52.871604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.871247    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy"
Oct 30 13:20:52.876433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.876410    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:52.916956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:52.916931    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"]
Oct 30 13:20:53.030176 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.030146    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.030371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.030307    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.030371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.030352    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.030371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.030375    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qw9r\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-kube-api-access-7qw9r\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.030511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.030441    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.030511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.030471    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.030574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.030527    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.030607 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.030598    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.030664 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.030652    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132350 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132259    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132382    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132406    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132425    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qw9r\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-kube-api-access-7qw9r\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132445    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132477    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132554    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132579    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132630    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132889 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132841    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-data\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.132951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132899    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.133009 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.132982    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-certs\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.133068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.133030    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-credential-socket\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.133261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.133241    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istiod-ca-cert\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.134496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.134479    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-envoy\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.134763 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.134746    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-podinfo\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.154839 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.154813    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-token\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.154925 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.154889    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qw9r\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-kube-api-access-7qw9r\") pod \"e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") " pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.190445 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.190424    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:53.515096 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.515070    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"]
Oct 30 13:20:53.526722 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:20:53.516969    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c7a654_01ce_4684_b4a4_59a62c26a26d.slice/crio-451cb23e5b6cdb5f684dce1a1dbcb6169bc5e29609770b22775fb5ec10e64235 WatchSource:0}: Error finding container 451cb23e5b6cdb5f684dce1a1dbcb6169bc5e29609770b22775fb5ec10e64235: Status 404 returned error can't find the container with id 451cb23e5b6cdb5f684dce1a1dbcb6169bc5e29609770b22775fb5ec10e64235
Oct 30 13:20:53.526722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.519753    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:20:53.526722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.519829    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:20:53.526722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.519873    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:20:53.534967 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.534935    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" event={"ID":"d1c7a654-01ce-4684-b4a4-59a62c26a26d","Type":"ContainerStarted","Data":"451cb23e5b6cdb5f684dce1a1dbcb6169bc5e29609770b22775fb5ec10e64235"}
Oct 30 13:20:53.604842 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:53.604792    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:54.197952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.197922    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:20:54.259300 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.259257    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:20:54.261980 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.261964    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:20:54.366166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366088    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-kube-api-access-hbzh5\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-kube-api-access-hbzh5\") pod \"73e67102-b858-4c96-ba57-e72d7a647fa2\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") "
Oct 30 13:20:54.366166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366123    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-kube-api-access-vgh9c\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-kube-api-access-vgh9c\") pod \"a940310d-3db1-4c9c-bf94-399db4eab1a3\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") "
Oct 30 13:20:54.366166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366152    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-socket\") pod \"a940310d-3db1-4c9c-bf94-399db4eab1a3\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") "
Oct 30 13:20:54.366406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366201    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-data\") pod \"e520d651-7215-4f0d-9ecd-c8eace846b1a\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") "
Oct 30 13:20:54.366406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366227    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-certs\") pod \"73e67102-b858-4c96-ba57-e72d7a647fa2\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") "
Oct 30 13:20:54.366406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366255    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-envoy\") pod \"e520d651-7215-4f0d-9ecd-c8eace846b1a\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") "
Oct 30 13:20:54.366406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366321    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-envoy\") pod \"73e67102-b858-4c96-ba57-e72d7a647fa2\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") "
Oct 30 13:20:54.366406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366367    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-data\") pod \"73e67102-b858-4c96-ba57-e72d7a647fa2\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") "
Oct 30 13:20:54.366627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366445    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-podinfo\") pod \"e520d651-7215-4f0d-9ecd-c8eace846b1a\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") "
Oct 30 13:20:54.366627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366473    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-token\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-token\") pod \"a940310d-3db1-4c9c-bf94-399db4eab1a3\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") "
Oct 30 13:20:54.366627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366504    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73e67102-b858-4c96-ba57-e72d7a647fa2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73e67102-b858-4c96-ba57-e72d7a647fa2-istiod-ca-cert\") pod \"73e67102-b858-4c96-ba57-e72d7a647fa2\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") "
Oct 30 13:20:54.366627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366546    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-kube-api-access-zbww7\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-kube-api-access-zbww7\") pod \"e520d651-7215-4f0d-9ecd-c8eace846b1a\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") "
Oct 30 13:20:54.366627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366605    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a940310d-3db1-4c9c-bf94-399db4eab1a3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a940310d-3db1-4c9c-bf94-399db4eab1a3-istiod-ca-cert\") pod \"a940310d-3db1-4c9c-bf94-399db4eab1a3\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") "
Oct 30 13:20:54.366860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366636    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-token\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-token\") pod \"73e67102-b858-4c96-ba57-e72d7a647fa2\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") "
Oct 30 13:20:54.366860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366673    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-credential-socket\") pod \"73e67102-b858-4c96-ba57-e72d7a647fa2\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") "
Oct 30 13:20:54.366860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366705    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-token\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-token\") pod \"e520d651-7215-4f0d-9ecd-c8eace846b1a\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") "
Oct 30 13:20:54.366860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366769    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-socket\") pod \"73e67102-b858-4c96-ba57-e72d7a647fa2\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") "
Oct 30 13:20:54.366860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366811    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-credential-socket\") pod \"a940310d-3db1-4c9c-bf94-399db4eab1a3\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") "
Oct 30 13:20:54.366860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366849    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-credential-socket\") pod \"e520d651-7215-4f0d-9ecd-c8eace846b1a\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") "
Oct 30 13:20:54.367123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366858    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-data" pod "e520d651-7215-4f0d-9ecd-c8eace846b1a" (UID: "e520d651-7215-4f0d-9ecd-c8eace846b1a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.367123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366908    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-socket\") pod \"e520d651-7215-4f0d-9ecd-c8eace846b1a\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") "
Oct 30 13:20:54.367123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.366959    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-certs\") pod \"e520d651-7215-4f0d-9ecd-c8eace846b1a\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") "
Oct 30 13:20:54.367123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367010    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-certs\") pod \"a940310d-3db1-4c9c-bf94-399db4eab1a3\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") "
Oct 30 13:20:54.367123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367043    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-podinfo\") pod \"73e67102-b858-4c96-ba57-e72d7a647fa2\" (UID: \"73e67102-b858-4c96-ba57-e72d7a647fa2\") "
Oct 30 13:20:54.367123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367083    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-data\") pod \"a940310d-3db1-4c9c-bf94-399db4eab1a3\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") "
Oct 30 13:20:54.367123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367067    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-certs" pod "73e67102-b858-4c96-ba57-e72d7a647fa2" (UID: "73e67102-b858-4c96-ba57-e72d7a647fa2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.367123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367106    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-socket" pod "a940310d-3db1-4c9c-bf94-399db4eab1a3" (UID: "a940310d-3db1-4c9c-bf94-399db4eab1a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.367560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367142    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-podinfo\") pod \"a940310d-3db1-4c9c-bf94-399db4eab1a3\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") "
Oct 30 13:20:54.367560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367174    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e520d651-7215-4f0d-9ecd-c8eace846b1a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e520d651-7215-4f0d-9ecd-c8eace846b1a-istiod-ca-cert\") pod \"e520d651-7215-4f0d-9ecd-c8eace846b1a\" (UID: \"e520d651-7215-4f0d-9ecd-c8eace846b1a\") "
Oct 30 13:20:54.367560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367207    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-socket" pod "73e67102-b858-4c96-ba57-e72d7a647fa2" (UID: "73e67102-b858-4c96-ba57-e72d7a647fa2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.367560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367222    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-envoy\") pod \"a940310d-3db1-4c9c-bf94-399db4eab1a3\" (UID: \"a940310d-3db1-4c9c-bf94-399db4eab1a3\") "
Oct 30 13:20:54.367560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367324    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-certs" pod "e520d651-7215-4f0d-9ecd-c8eace846b1a" (UID: "e520d651-7215-4f0d-9ecd-c8eace846b1a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.367560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367520    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-credential-socket" pod "a940310d-3db1-4c9c-bf94-399db4eab1a3" (UID: "a940310d-3db1-4c9c-bf94-399db4eab1a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.367891 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367866    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.367949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367897    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.367949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367911    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.367949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367924    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.367949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367937    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.368144 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.367950    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.368195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.368148    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-credential-socket" pod "e520d651-7215-4f0d-9ecd-c8eace846b1a" (UID: "e520d651-7215-4f0d-9ecd-c8eace846b1a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.368375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.368352    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-socket" pod "e520d651-7215-4f0d-9ecd-c8eace846b1a" (UID: "e520d651-7215-4f0d-9ecd-c8eace846b1a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.368566 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.368545    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-data" pod "a940310d-3db1-4c9c-bf94-399db4eab1a3" (UID: "a940310d-3db1-4c9c-bf94-399db4eab1a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.369500 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.369471    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-podinfo" pod "e520d651-7215-4f0d-9ecd-c8eace846b1a" (UID: "e520d651-7215-4f0d-9ecd-c8eace846b1a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:54.369608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.369549    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-token" pod "a940310d-3db1-4c9c-bf94-399db4eab1a3" (UID: "a940310d-3db1-4c9c-bf94-399db4eab1a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.369702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.369656    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-data" pod "73e67102-b858-4c96-ba57-e72d7a647fa2" (UID: "73e67102-b858-4c96-ba57-e72d7a647fa2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.369761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.369695    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-envoy" pod "73e67102-b858-4c96-ba57-e72d7a647fa2" (UID: "73e67102-b858-4c96-ba57-e72d7a647fa2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.370326 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.370302    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73e67102-b858-4c96-ba57-e72d7a647fa2-istiod-ca-cert" pod "73e67102-b858-4c96-ba57-e72d7a647fa2" (UID: "73e67102-b858-4c96-ba57-e72d7a647fa2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:54.370543 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.370516    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-envoy" pod "e520d651-7215-4f0d-9ecd-c8eace846b1a" (UID: "e520d651-7215-4f0d-9ecd-c8eace846b1a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.370838 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.370810    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-credential-socket" pod "73e67102-b858-4c96-ba57-e72d7a647fa2" (UID: "73e67102-b858-4c96-ba57-e72d7a647fa2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.371223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.371191    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-certs" pod "a940310d-3db1-4c9c-bf94-399db4eab1a3" (UID: "a940310d-3db1-4c9c-bf94-399db4eab1a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.371787 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.371571    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-kube-api-access-vgh9c" pod "a940310d-3db1-4c9c-bf94-399db4eab1a3" (UID: "a940310d-3db1-4c9c-bf94-399db4eab1a3"). InnerVolumeSpecName "kube-api-access-vgh9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.371884 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.371788    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-podinfo" pod "73e67102-b858-4c96-ba57-e72d7a647fa2" (UID: "73e67102-b858-4c96-ba57-e72d7a647fa2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:54.372118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.372050    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e520d651-7215-4f0d-9ecd-c8eace846b1a-istiod-ca-cert" pod "e520d651-7215-4f0d-9ecd-c8eace846b1a" (UID: "e520d651-7215-4f0d-9ecd-c8eace846b1a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:54.372118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.372085    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a940310d-3db1-4c9c-bf94-399db4eab1a3-istiod-ca-cert" pod "a940310d-3db1-4c9c-bf94-399db4eab1a3" (UID: "a940310d-3db1-4c9c-bf94-399db4eab1a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:54.372392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.372345    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-token" pod "73e67102-b858-4c96-ba57-e72d7a647fa2" (UID: "73e67102-b858-4c96-ba57-e72d7a647fa2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.372756 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.372733    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-envoy" pod "a940310d-3db1-4c9c-bf94-399db4eab1a3" (UID: "a940310d-3db1-4c9c-bf94-399db4eab1a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:20:54.372960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.372941    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-token" pod "e520d651-7215-4f0d-9ecd-c8eace846b1a" (UID: "e520d651-7215-4f0d-9ecd-c8eace846b1a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.373520 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.373496    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-kube-api-access-zbww7" pod "e520d651-7215-4f0d-9ecd-c8eace846b1a" (UID: "e520d651-7215-4f0d-9ecd-c8eace846b1a"). InnerVolumeSpecName "kube-api-access-zbww7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.373598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.373516    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-podinfo" pod "a940310d-3db1-4c9c-bf94-399db4eab1a3" (UID: "a940310d-3db1-4c9c-bf94-399db4eab1a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:20:54.374423 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.374394    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-kube-api-access-hbzh5" pod "73e67102-b858-4c96-ba57-e72d7a647fa2" (UID: "73e67102-b858-4c96-ba57-e72d7a647fa2"). InnerVolumeSpecName "kube-api-access-hbzh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:54.468848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468814    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.468848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468842    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e520d651-7215-4f0d-9ecd-c8eace846b1a-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.468848 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468850    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468859    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbzh5\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-kube-api-access-hbzh5\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468869    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgh9c\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-kube-api-access-vgh9c\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468877    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468885    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468892    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468899    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468906    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468915    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73e67102-b858-4c96-ba57-e72d7a647fa2-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468922    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbww7\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-kube-api-access-zbww7\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468930    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a940310d-3db1-4c9c-bf94-399db4eab1a3-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468937    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468944    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73e67102-b858-4c96-ba57-e72d7a647fa2-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468959    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e520d651-7215-4f0d-9ecd-c8eace846b1a-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468967    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468974    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e520d651-7215-4f0d-9ecd-c8eace846b1a-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468983    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468991    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73e67102-b858-4c96-ba57-e72d7a647fa2-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.469101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.468998    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a940310d-3db1-4c9c-bf94-399db4eab1a3-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:54.550673 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.550633    5781 generic.go:356] "Generic (PLEG): container finished" podID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerID="1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1" exitCode=0
Oct 30 13:20:54.550864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.550688    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"
Oct 30 13:20:54.550864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.550719    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" event={"ID":"a940310d-3db1-4c9c-bf94-399db4eab1a3","Type":"ContainerDied","Data":"1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1"}
Oct 30 13:20:54.550864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.550817    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx" event={"ID":"a940310d-3db1-4c9c-bf94-399db4eab1a3","Type":"ContainerDied","Data":"71cb0bccf280e324a03a6ccd50bdd4b1bb75d610a6ea60e5abdc588608a5e257"}
Oct 30 13:20:54.550864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.550835    5781 scope.go:122] "RemoveContainer" containerID="1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1"
Oct 30 13:20:54.552556 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.552535    5781 generic.go:356] "Generic (PLEG): container finished" podID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerID="816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1" exitCode=0
Oct 30 13:20:54.552680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.552620    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" event={"ID":"e520d651-7215-4f0d-9ecd-c8eace846b1a","Type":"ContainerDied","Data":"816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1"}
Oct 30 13:20:54.552680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.552647    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"
Oct 30 13:20:54.552680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.552659    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb" event={"ID":"e520d651-7215-4f0d-9ecd-c8eace846b1a","Type":"ContainerDied","Data":"1dcccfb0f342998af83e111f8dc78cbf7d8835fe1a6ddcb0c86948219121e28f"}
Oct 30 13:20:54.555169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.555144    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" event={"ID":"d1c7a654-01ce-4684-b4a4-59a62c26a26d","Type":"ContainerStarted","Data":"308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c"}
Oct 30 13:20:54.556901 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.556844    5781 generic.go:356] "Generic (PLEG): container finished" podID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerID="d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e" exitCode=0
Oct 30 13:20:54.556901 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.556884    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" event={"ID":"73e67102-b858-4c96-ba57-e72d7a647fa2","Type":"ContainerDied","Data":"d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e"}
Oct 30 13:20:54.557051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.556936    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct" event={"ID":"73e67102-b858-4c96-ba57-e72d7a647fa2","Type":"ContainerDied","Data":"7ae76cefc9202e991594093ac20f254ae8603120a20db661889eb78699900b7b"}
Oct 30 13:20:54.557051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.556894    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"
Oct 30 13:20:54.563023 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.563009    5781 scope.go:122] "RemoveContainer" containerID="1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1"
Oct 30 13:20:54.563390 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:54.563340    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1\": container with ID starting with 1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1 not found: ID does not exist" containerID="1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1"
Oct 30 13:20:54.563390 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.563374    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1"} err="failed to get container status \"1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1\": rpc error: code = NotFound desc = could not find container \"1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1\": container with ID starting with 1a7e76d6dd5f4a41292b15812627cd06a3227c1254ac7cc74a171cd312e519b1 not found: ID does not exist"
Oct 30 13:20:54.563540 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.563398    5781 scope.go:122] "RemoveContainer" containerID="816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1"
Oct 30 13:20:54.574079 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.574063    5781 scope.go:122] "RemoveContainer" containerID="96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e"
Oct 30 13:20:54.582953 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.582919    5781 scope.go:122] "RemoveContainer" containerID="816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1"
Oct 30 13:20:54.583315 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:54.583234    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1\": container with ID starting with 816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1 not found: ID does not exist" containerID="816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1"
Oct 30 13:20:54.583436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.583267    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1"} err="failed to get container status \"816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1\": rpc error: code = NotFound desc = could not find container \"816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1\": container with ID starting with 816ce73f235ceb03501b6de8e2adc2118a70fdbc18a3a364223a2b3b6cb110c1 not found: ID does not exist"
Oct 30 13:20:54.583436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.583425    5781 scope.go:122] "RemoveContainer" containerID="96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e"
Oct 30 13:20:54.583792 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:54.583774    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e\": container with ID starting with 96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e not found: ID does not exist" containerID="96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e"
Oct 30 13:20:54.583880 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.583796    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e"} err="failed to get container status \"96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e\": rpc error: code = NotFound desc = could not find container \"96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e\": container with ID starting with 96c1ef9ed2622f6f11378932e390c23b0164b3fa9e9c377cf35e748fbfb0671e not found: ID does not exist"
Oct 30 13:20:54.583880 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.583810    5781 scope.go:122] "RemoveContainer" containerID="d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e"
Oct 30 13:20:54.592745 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.592730    5781 scope.go:122] "RemoveContainer" containerID="d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e"
Oct 30 13:20:54.592988 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:20:54.592968    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e\": container with ID starting with d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e not found: ID does not exist" containerID="d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e"
Oct 30 13:20:54.593030 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.592994    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e"} err="failed to get container status \"d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e\": rpc error: code = NotFound desc = could not find container \"d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e\": container with ID starting with d716e0c4576a57423bf58c3b82f2d3c16523694256f86e814a28c8bad5fd2f3e not found: ID does not exist"
Oct 30 13:20:54.607253 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.607229    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"]
Oct 30 13:20:54.611876 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.611859    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-sdkdb"]
Oct 30 13:20:54.659224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.659126    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" podStartSLOduration=2.659110688 podStartE2EDuration="2.659110688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:54.655649557 +0000 UTC m=+1391.952867100" watchObservedRunningTime="2026-10-30 13:20:54.659110688 +0000 UTC m=+1391.956328230"
Oct 30 13:20:54.741440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.741400    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"]
Oct 30 13:20:54.744786 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.744762    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-kkkxx"]
Oct 30 13:20:54.766355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.766329    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"]
Oct 30 13:20:54.770119 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.770097    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d0a18c-openshift-default-547d57db97-2ztct"]
Oct 30 13:20:54.845822 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.845794    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" path="/var/lib/kubelet/pods/73e67102-b858-4c96-ba57-e72d7a647fa2/volumes"
Oct 30 13:20:54.846256 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.846243    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" path="/var/lib/kubelet/pods/a940310d-3db1-4c9c-bf94-399db4eab1a3/volumes"
Oct 30 13:20:54.846795 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:54.846723    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" path="/var/lib/kubelet/pods/e520d651-7215-4f0d-9ecd-c8eace846b1a/volumes"
Oct 30 13:20:55.045723 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.045689    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"]
Oct 30 13:20:55.046282 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046263    5781 cpu_manager.go:414] "containerMap: removing container" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy"
Oct 30 13:20:55.046397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046297    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy"
Oct 30 13:20:55.046397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046312    5781 cpu_manager.go:414] "containerMap: removing container" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy"
Oct 30 13:20:55.046397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046319    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy"
Oct 30 13:20:55.046397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046339    5781 cpu_manager.go:414] "containerMap: removing container" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy"
Oct 30 13:20:55.046397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046347    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="519f908a-bf41-4285-8ac4-1abe0a9d893e" containerName="istio-proxy"
Oct 30 13:20:55.046397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046379    5781 cpu_manager.go:414] "containerMap: removing container" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy"
Oct 30 13:20:55.046397 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046387    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy"
Oct 30 13:20:55.046746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046408    5781 cpu_manager.go:414] "containerMap: removing container" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy"
Oct 30 13:20:55.046746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046421    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy"
Oct 30 13:20:55.046746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046561    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy"
Oct 30 13:20:55.046746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046580    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="73e67102-b858-4c96-ba57-e72d7a647fa2" containerName="istio-proxy"
Oct 30 13:20:55.046746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046588    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="a940310d-3db1-4c9c-bf94-399db4eab1a3" containerName="istio-proxy"
Oct 30 13:20:55.046746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.046603    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="e520d651-7215-4f0d-9ecd-c8eace846b1a" containerName="istio-proxy"
Oct 30 13:20:55.052033 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.052011    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.067423 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.065012    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"]
Oct 30 13:20:55.138370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.138181    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:20:55.138370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.138271    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:20:55.176052 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.176023    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jb7x\" (UniqueName: \"kubernetes.io/projected/0727e035-c05f-4655-a822-6057e6d2ddb0-kube-api-access-6jb7x\") pod \"maas-api-revoke-sub-8951bdfd-qbkpp\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.176230 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.176140    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8951bdfd-qbkpp\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.176230 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.176202    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8951bdfd-qbkpp\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.190695 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.190676    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:20:55.192368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.192346    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 13:20:55.192437 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.192391    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 13:20:55.276957 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.276925    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jb7x\" (UniqueName: \"kubernetes.io/projected/0727e035-c05f-4655-a822-6057e6d2ddb0-kube-api-access-6jb7x\") pod \"maas-api-revoke-sub-8951bdfd-qbkpp\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.277394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.276991    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8951bdfd-qbkpp\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.277394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.277036    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8951bdfd-qbkpp\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.277657 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.277635    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:55.277737 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.277721    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8951bdfd-qbkpp\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.279437 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.279417    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8951bdfd-qbkpp\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.312118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.312064    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jb7x\" (UniqueName: \"kubernetes.io/projected/0727e035-c05f-4655-a822-6057e6d2ddb0-kube-api-access-6jb7x\") pod \"maas-api-revoke-sub-8951bdfd-qbkpp\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.368400 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.368374    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:55.547488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.547348    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:55.694094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:55.694074    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"]
Oct 30 13:20:55.696184 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:20:55.696163    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0727e035_c05f_4655_a822_6057e6d2ddb0.slice/crio-4f1817dc4d9e84a302dc5732c907e20dcca125065966804ad0b27c255c048183 WatchSource:0}: Error finding container 4f1817dc4d9e84a302dc5732c907e20dcca125065966804ad0b27c255c048183: Status 404 returned error can't find the container with id 4f1817dc4d9e84a302dc5732c907e20dcca125065966804ad0b27c255c048183
Oct 30 13:20:56.191560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:56.191530    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 13:20:56.191720 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:56.191591    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 13:20:56.575235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:56.575135    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp" event={"ID":"0727e035-c05f-4655-a822-6057e6d2ddb0","Type":"ContainerStarted","Data":"ab2f4b44ad86cc64daf008b9739b4f14458aac02913b966040aeaba491f81e61"}
Oct 30 13:20:56.575235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:56.575202    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp" event={"ID":"0727e035-c05f-4655-a822-6057e6d2ddb0","Type":"ContainerStarted","Data":"4f1817dc4d9e84a302dc5732c907e20dcca125065966804ad0b27c255c048183"}
Oct 30 13:20:57.049038 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:57.048964    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp" podStartSLOduration=2.048945881 podStartE2EDuration="2.048945881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:20:56.638504381 +0000 UTC m=+1393.935721924" watchObservedRunningTime="2026-10-30 13:20:57.048945881 +0000 UTC m=+1394.346163422"
Oct 30 13:20:57.049669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:57.049638    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"]
Oct 30 13:20:57.052713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:57.052687    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0c5315f2-qj5bs"]
Oct 30 13:20:57.158433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:57.158374    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:57.191224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:57.191186    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 13:20:57.191382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:57.191247    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 13:20:57.583908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:57.583876    5781 generic.go:356] "Generic (PLEG): container finished" podID="0727e035-c05f-4655-a822-6057e6d2ddb0" containerID="ab2f4b44ad86cc64daf008b9739b4f14458aac02913b966040aeaba491f81e61" exitCode=0
Oct 30 13:20:57.584387 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:57.583973    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp" event={"ID":"0727e035-c05f-4655-a822-6057e6d2ddb0","Type":"ContainerDied","Data":"ab2f4b44ad86cc64daf008b9739b4f14458aac02913b966040aeaba491f81e61"}
Oct 30 13:20:58.045710 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.045668    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"]
Oct 30 13:20:58.051655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.051625    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8c7f8b52-6x5d9"]
Oct 30 13:20:58.191645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.191605    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 13:20:58.191816 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.191661    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 13:20:58.772328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.769094    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:20:58.778008 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.777984    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:58.848882 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.848765    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-xks-ca\") pod \"0727e035-c05f-4655-a822-6057e6d2ddb0\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") "
Oct 30 13:20:58.849113 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.848893    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0727e035-c05f-4655-a822-6057e6d2ddb0-kube-api-access-6jb7x\" (UniqueName: \"kubernetes.io/projected/0727e035-c05f-4655-a822-6057e6d2ddb0-kube-api-access-6jb7x\") pod \"0727e035-c05f-4655-a822-6057e6d2ddb0\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") "
Oct 30 13:20:58.849113 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.848997    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-openshift-ca\") pod \"0727e035-c05f-4655-a822-6057e6d2ddb0\" (UID: \"0727e035-c05f-4655-a822-6057e6d2ddb0\") "
Oct 30 13:20:58.849710 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.849684    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-openshift-ca" pod "0727e035-c05f-4655-a822-6057e6d2ddb0" (UID: "0727e035-c05f-4655-a822-6057e6d2ddb0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:20:58.852446 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.852425    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0727e035-c05f-4655-a822-6057e6d2ddb0-kube-api-access-6jb7x" pod "0727e035-c05f-4655-a822-6057e6d2ddb0" (UID: "0727e035-c05f-4655-a822-6057e6d2ddb0"). InnerVolumeSpecName "kube-api-access-6jb7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:20:58.856420 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.856393    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-xks-ca" pod "0727e035-c05f-4655-a822-6057e6d2ddb0" (UID: "0727e035-c05f-4655-a822-6057e6d2ddb0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:20:58.863242 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.863209    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="652d2429-a2d4-4e03-bc92-bf7b227b8b4c" path="/var/lib/kubelet/pods/652d2429-a2d4-4e03-bc92-bf7b227b8b4c/volumes"
Oct 30 13:20:58.863976 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.863956    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f" path="/var/lib/kubelet/pods/f454b33e-bfcc-49bb-b2e9-2cc8ed59a54f/volumes"
Oct 30 13:20:58.950466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.950429    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jb7x\" (UniqueName: \"kubernetes.io/projected/0727e035-c05f-4655-a822-6057e6d2ddb0-kube-api-access-6jb7x\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:58.950466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.950456    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:58.950466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:58.950467    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0727e035-c05f-4655-a822-6057e6d2ddb0-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:20:59.191179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:59.191145    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 13:20:59.191370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:59.191195    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 13:20:59.599764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:59.599671    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"
Oct 30 13:20:59.599764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:59.599682    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp" event={"ID":"0727e035-c05f-4655-a822-6057e6d2ddb0","Type":"ContainerDied","Data":"4f1817dc4d9e84a302dc5732c907e20dcca125065966804ad0b27c255c048183"}
Oct 30 13:20:59.599764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:20:59.599735    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f1817dc4d9e84a302dc5732c907e20dcca125065966804ad0b27c255c048183"
Oct 30 13:21:00.191259 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:00.191224    5781 patch_prober.go:29] interesting pod/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused" start-of-body=
Oct 30 13:21:00.191693 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:00.191281    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.111:15021/healthz/ready\": dial tcp 10.134.0.111:15021: connect: connection refused"
Oct 30 13:21:00.501269 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:00.501171    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:00.727392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:00.727344    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" containerID="cri-o://c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b" gracePeriod=45
Oct 30 13:21:00.746167 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:00.746123    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" containerID="cri-o://fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb" gracePeriod=45
Oct 30 13:21:00.809718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:00.809687    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main" containerID="cri-o://5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952" gracePeriod=45
Oct 30 13:21:01.003570 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.003154    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.60:8000/ready\": dial tcp 10.134.0.60:8000: connect: connection refused"
Oct 30 13:21:01.069164 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.069114    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"]
Oct 30 13:21:01.079240 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.079124    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:21:01.086783 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.086761    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:21:01.119355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.119329    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:21:01.177099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177068    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-home\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-home\") pod \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") "
Oct 30 13:21:01.177248 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177161    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4f7d4f7-9ba6-4782-9498-902b09b910f8-kube-api-access-5p6dc\" (UniqueName: \"kubernetes.io/projected/d4f7d4f7-9ba6-4782-9498-902b09b910f8-kube-api-access-5p6dc\") pod \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") "
Oct 30 13:21:01.177248 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177207    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-model-cache\") pod \"1f5098eb-9409-4853-a343-8beeec46ab48\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") "
Oct 30 13:21:01.177248 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177242    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-dshm\") pod \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") "
Oct 30 13:21:01.177433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177337    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4969603-8e31-49dd-ada8-ac59276f30b2-kube-api-access-pfmrr\" (UniqueName: \"kubernetes.io/projected/e4969603-8e31-49dd-ada8-ac59276f30b2-kube-api-access-pfmrr\") pod \"e4969603-8e31-49dd-ada8-ac59276f30b2\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") "
Oct 30 13:21:01.177433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177376    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-home" pod "d4f7d4f7-9ba6-4782-9498-902b09b910f8" (UID: "d4f7d4f7-9ba6-4782-9498-902b09b910f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.177433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177385    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e4969603-8e31-49dd-ada8-ac59276f30b2-tls-certs\" (UniqueName: \"kubernetes.io/secret/e4969603-8e31-49dd-ada8-ac59276f30b2-tls-certs\") pod \"e4969603-8e31-49dd-ada8-ac59276f30b2\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") "
Oct 30 13:21:01.177591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177453    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-dshm\") pod \"e4969603-8e31-49dd-ada8-ac59276f30b2\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") "
Oct 30 13:21:01.177591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177490    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f5098eb-9409-4853-a343-8beeec46ab48-kube-api-access-ls8kg\" (UniqueName: \"kubernetes.io/projected/1f5098eb-9409-4853-a343-8beeec46ab48-kube-api-access-ls8kg\") pod \"1f5098eb-9409-4853-a343-8beeec46ab48\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") "
Oct 30 13:21:01.177591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177512    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-tmp-dir\") pod \"e4969603-8e31-49dd-ada8-ac59276f30b2\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") "
Oct 30 13:21:01.177591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177556    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-dshm\") pod \"1f5098eb-9409-4853-a343-8beeec46ab48\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") "
Oct 30 13:21:01.177801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177595    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tmp-dir\") pod \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") "
Oct 30 13:21:01.177801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177646    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-home\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-home\") pod \"1f5098eb-9409-4853-a343-8beeec46ab48\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") "
Oct 30 13:21:01.177801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177690    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f5098eb-9409-4853-a343-8beeec46ab48-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f5098eb-9409-4853-a343-8beeec46ab48-tls-certs\") pod \"1f5098eb-9409-4853-a343-8beeec46ab48\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") "
Oct 30 13:21:01.177801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177731    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-tmp-dir\") pod \"1f5098eb-9409-4853-a343-8beeec46ab48\" (UID: \"1f5098eb-9409-4853-a343-8beeec46ab48\") "
Oct 30 13:21:01.177801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177760    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-model-cache\") pod \"e4969603-8e31-49dd-ada8-ac59276f30b2\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") "
Oct 30 13:21:01.178061 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177839    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-model-cache\") pod \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") "
Oct 30 13:21:01.178061 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177864    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-home\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-home\") pod \"e4969603-8e31-49dd-ada8-ac59276f30b2\" (UID: \"e4969603-8e31-49dd-ada8-ac59276f30b2\") "
Oct 30 13:21:01.178061 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.177906    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tls-certs\" (UniqueName: \"kubernetes.io/secret/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tls-certs\") pod \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\" (UID: \"d4f7d4f7-9ba6-4782-9498-902b09b910f8\") "
Oct 30 13:21:01.178542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.178515    5781 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-home\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.178657 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.178570    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-model-cache" pod "1f5098eb-9409-4853-a343-8beeec46ab48" (UID: "1f5098eb-9409-4853-a343-8beeec46ab48"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.180856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.180824    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-model-cache" pod "e4969603-8e31-49dd-ada8-ac59276f30b2" (UID: "e4969603-8e31-49dd-ada8-ac59276f30b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.181013 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.180991    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-tmp-dir" pod "e4969603-8e31-49dd-ada8-ac59276f30b2" (UID: "e4969603-8e31-49dd-ada8-ac59276f30b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.181067 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.181027    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-tmp-dir" pod "1f5098eb-9409-4853-a343-8beeec46ab48" (UID: "1f5098eb-9409-4853-a343-8beeec46ab48"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.181252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.181225    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-model-cache" pod "d4f7d4f7-9ba6-4782-9498-902b09b910f8" (UID: "d4f7d4f7-9ba6-4782-9498-902b09b910f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.182935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.181516    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tmp-dir" pod "d4f7d4f7-9ba6-4782-9498-902b09b910f8" (UID: "d4f7d4f7-9ba6-4782-9498-902b09b910f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.182935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.181560    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-home" pod "1f5098eb-9409-4853-a343-8beeec46ab48" (UID: "1f5098eb-9409-4853-a343-8beeec46ab48"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.182935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.181852    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tls-certs" pod "d4f7d4f7-9ba6-4782-9498-902b09b910f8" (UID: "d4f7d4f7-9ba6-4782-9498-902b09b910f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:01.182935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.182870    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4969603-8e31-49dd-ada8-ac59276f30b2-kube-api-access-pfmrr" pod "e4969603-8e31-49dd-ada8-ac59276f30b2" (UID: "e4969603-8e31-49dd-ada8-ac59276f30b2"). InnerVolumeSpecName "kube-api-access-pfmrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:01.183179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.183014    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-dshm" pod "1f5098eb-9409-4853-a343-8beeec46ab48" (UID: "1f5098eb-9409-4853-a343-8beeec46ab48"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.186731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.183441    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-home" pod "e4969603-8e31-49dd-ada8-ac59276f30b2" (UID: "e4969603-8e31-49dd-ada8-ac59276f30b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.198328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.194777    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f5098eb-9409-4853-a343-8beeec46ab48-kube-api-access-ls8kg" pod "1f5098eb-9409-4853-a343-8beeec46ab48" (UID: "1f5098eb-9409-4853-a343-8beeec46ab48"). InnerVolumeSpecName "kube-api-access-ls8kg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:01.198328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.194894    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-dshm" pod "e4969603-8e31-49dd-ada8-ac59276f30b2" (UID: "e4969603-8e31-49dd-ada8-ac59276f30b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.198328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.194973    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4969603-8e31-49dd-ada8-ac59276f30b2-tls-certs" pod "e4969603-8e31-49dd-ada8-ac59276f30b2" (UID: "e4969603-8e31-49dd-ada8-ac59276f30b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:01.198328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.195059    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f7d4f7-9ba6-4782-9498-902b09b910f8-kube-api-access-5p6dc" pod "d4f7d4f7-9ba6-4782-9498-902b09b910f8" (UID: "d4f7d4f7-9ba6-4782-9498-902b09b910f8"). InnerVolumeSpecName "kube-api-access-5p6dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:01.199031 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.199004    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:21:01.200017 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.199985    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:21:01.200671 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.200647    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-dshm" pod "d4f7d4f7-9ba6-4782-9498-902b09b910f8" (UID: "d4f7d4f7-9ba6-4782-9498-902b09b910f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:01.200761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.200690    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:21:01.201589 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.201565    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f5098eb-9409-4853-a343-8beeec46ab48-tls-certs" pod "1f5098eb-9409-4853-a343-8beeec46ab48" (UID: "1f5098eb-9409-4853-a343-8beeec46ab48"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:01.280161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280129    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfmrr\" (UniqueName: \"kubernetes.io/projected/e4969603-8e31-49dd-ada8-ac59276f30b2-kube-api-access-pfmrr\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280161    5781 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4969603-8e31-49dd-ada8-ac59276f30b2-tls-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280178    5781 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-dshm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280191    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls8kg\" (UniqueName: \"kubernetes.io/projected/1f5098eb-9409-4853-a343-8beeec46ab48-kube-api-access-ls8kg\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280205    5781 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-tmp-dir\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280219    5781 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-dshm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280231    5781 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tmp-dir\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280243    5781 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-home\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280254    5781 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f5098eb-9409-4853-a343-8beeec46ab48-tls-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280266    5781 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-tmp-dir\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280278    5781 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-model-cache\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280304    5781 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-model-cache\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280316    5781 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e4969603-8e31-49dd-ada8-ac59276f30b2-home\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280328    5781 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4f7d4f7-9ba6-4782-9498-902b09b910f8-tls-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280340    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p6dc\" (UniqueName: \"kubernetes.io/projected/d4f7d4f7-9ba6-4782-9498-902b09b910f8-kube-api-access-5p6dc\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280352    5781 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f5098eb-9409-4853-a343-8beeec46ab48-model-cache\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.280381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.280364    5781 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d4f7-9ba6-4782-9498-902b09b910f8-dshm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:01.362558 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.362396    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:01.540378 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.540345    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:01.615829 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.615748    5781 generic.go:356] "Generic (PLEG): container finished" podID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerID="c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b" exitCode=0
Oct 30 13:21:01.615829 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.615804    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"
Oct 30 13:21:01.616041 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.615827    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" event={"ID":"e4969603-8e31-49dd-ada8-ac59276f30b2","Type":"ContainerDied","Data":"c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b"}
Oct 30 13:21:01.616041 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.615867    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" event={"ID":"e4969603-8e31-49dd-ada8-ac59276f30b2","Type":"ContainerDied","Data":"983a24a3136fdd073ca14a47cf57a144f862e497b77fe70ba1f72d9acc87ce2c"}
Oct 30 13:21:01.616041 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.615890    5781 scope.go:122] "RemoveContainer" containerID="c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b"
Oct 30 13:21:01.617350 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.617327    5781 generic.go:356] "Generic (PLEG): container finished" podID="1f5098eb-9409-4853-a343-8beeec46ab48" containerID="5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952" exitCode=0
Oct 30 13:21:01.617468 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.617402    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" event={"ID":"1f5098eb-9409-4853-a343-8beeec46ab48","Type":"ContainerDied","Data":"5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952"}
Oct 30 13:21:01.617468 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.617428    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm" event={"ID":"1f5098eb-9409-4853-a343-8beeec46ab48","Type":"ContainerDied","Data":"5009de971f3246ca2031d882c7d2068c8f7c8d2ae17e19caea16234b64729327"}
Oct 30 13:21:01.617675 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.617652    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"
Oct 30 13:21:01.622638 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.622614    5781 generic.go:356] "Generic (PLEG): container finished" podID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerID="fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb" exitCode=0
Oct 30 13:21:01.626381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.626358    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"
Oct 30 13:21:01.627296 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.627256    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" event={"ID":"d4f7d4f7-9ba6-4782-9498-902b09b910f8","Type":"ContainerDied","Data":"fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb"}
Oct 30 13:21:01.627398 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.627308    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb" event={"ID":"d4f7d4f7-9ba6-4782-9498-902b09b910f8","Type":"ContainerDied","Data":"a2a004e6a7ca65dcc2cd4b0c1d182e897b372b6776aab8d68b175a35f8ffe7d5"}
Oct 30 13:21:01.637344 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.637325    5781 scope.go:122] "RemoveContainer" containerID="c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b"
Oct 30 13:21:01.637634 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:01.637612    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b\": container with ID starting with c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b not found: ID does not exist" containerID="c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b"
Oct 30 13:21:01.637727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.637645    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b"} err="failed to get container status \"c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b\": rpc error: code = NotFound desc = could not find container \"c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b\": container with ID starting with c003e6304510ba3ef41e2227d60b6e01c1be34386cd8c064e3e5a19da930f31b not found: ID does not exist"
Oct 30 13:21:01.637727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.637667    5781 scope.go:122] "RemoveContainer" containerID="5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952"
Oct 30 13:21:01.647043 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.647024    5781 scope.go:122] "RemoveContainer" containerID="5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952"
Oct 30 13:21:01.647315 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:01.647297    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952\": container with ID starting with 5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952 not found: ID does not exist" containerID="5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952"
Oct 30 13:21:01.647388 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.647322    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952"} err="failed to get container status \"5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952\": rpc error: code = NotFound desc = could not find container \"5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952\": container with ID starting with 5b6888142a5d02167ab5029ca9edd56273404c5633e17514875cd4f98e793952 not found: ID does not exist"
Oct 30 13:21:01.647388 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.647337    5781 scope.go:122] "RemoveContainer" containerID="fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb"
Oct 30 13:21:01.655717 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.655698    5781 scope.go:122] "RemoveContainer" containerID="fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb"
Oct 30 13:21:01.655967 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:01.655947    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb\": container with ID starting with fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb not found: ID does not exist" containerID="fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb"
Oct 30 13:21:01.656032 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.655976    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb"} err="failed to get container status \"fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb\": rpc error: code = NotFound desc = could not find container \"fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb\": container with ID starting with fbb8a28f16f70840b468d43735c1ffa2c235bd31504214ca75ce9f912d3d6feb not found: ID does not exist"
Oct 30 13:21:01.672115 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.672095    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"]
Oct 30 13:21:01.675997 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.675979    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/simulator-w2-f76c09-kserve-bf97b8948-gdscm"]
Oct 30 13:21:01.726829 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.726806    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"]
Oct 30 13:21:01.732464 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.732446    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh"]
Oct 30 13:21:01.767368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.767345    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"]
Oct 30 13:21:01.775491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.775473    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-f76c09/premium-w2-f76c09-kserve-76c557b99c-th2xb"]
Oct 30 13:21:01.996080 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:01.996040    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-f76c09/distinct-w2-f76c09-kserve-5d7f4f648d-4vpwh" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.104:8000/ready\": context deadline exceeded"
Oct 30 13:21:01.999437 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:01.999410    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:21:02.410704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:02.410615    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:02.844780 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:02.844737    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" path="/var/lib/kubelet/pods/1f5098eb-9409-4853-a343-8beeec46ab48/volumes"
Oct 30 13:21:02.845375 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:02.845352    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" path="/var/lib/kubelet/pods/d4f7d4f7-9ba6-4782-9498-902b09b910f8/volumes"
Oct 30 13:21:02.845925 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:02.845899    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" path="/var/lib/kubelet/pods/e4969603-8e31-49dd-ada8-ac59276f30b2/volumes"
Oct 30 13:21:03.816409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.816376    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"]
Oct 30 13:21:03.816877 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.816861    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0727e035-c05f-4655-a822-6057e6d2ddb0" containerName="revoke-keys"
Oct 30 13:21:03.816877 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.816872    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0727e035-c05f-4655-a822-6057e6d2ddb0" containerName="revoke-keys"
Oct 30 13:21:03.816968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.816892    5781 cpu_manager.go:414] "containerMap: removing container" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main"
Oct 30 13:21:03.816968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.816897    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main"
Oct 30 13:21:03.816968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.816912    5781 cpu_manager.go:414] "containerMap: removing container" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main"
Oct 30 13:21:03.816968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.816918    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main"
Oct 30 13:21:03.816968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.816932    5781 cpu_manager.go:414] "containerMap: removing container" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main"
Oct 30 13:21:03.816968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.816936    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main"
Oct 30 13:21:03.817153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.817024    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4f7d4f7-9ba6-4782-9498-902b09b910f8" containerName="main"
Oct 30 13:21:03.817153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.817035    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f5098eb-9409-4853-a343-8beeec46ab48" containerName="main"
Oct 30 13:21:03.817153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.817044    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4969603-8e31-49dd-ada8-ac59276f30b2" containerName="main"
Oct 30 13:21:03.817153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.817053    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0727e035-c05f-4655-a822-6057e6d2ddb0" containerName="revoke-keys"
Oct 30 13:21:03.821781 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.821756    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:03.848703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.848683    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:21:03.848827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.848732    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:21:03.853454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.853433    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"]
Oct 30 13:21:03.917992 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.917952    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fce879a9-7f7js\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:03.918178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.917994    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhmx9\" (UniqueName: \"kubernetes.io/projected/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-kube-api-access-bhmx9\") pod \"maas-api-revoke-sub-fce879a9-7f7js\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:03.918178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:03.918095    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fce879a9-7f7js\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:04.020063 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.020024    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fce879a9-7f7js\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:04.020247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.020071    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhmx9\" (UniqueName: \"kubernetes.io/projected/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-kube-api-access-bhmx9\") pod \"maas-api-revoke-sub-fce879a9-7f7js\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:04.020247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.020107    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fce879a9-7f7js\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:04.020807 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.020779    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-fce879a9-7f7js\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:04.022484 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.022461    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-fce879a9-7f7js\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:04.043444 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.043422    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhmx9\" (UniqueName: \"kubernetes.io/projected/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-kube-api-access-bhmx9\") pod \"maas-api-revoke-sub-fce879a9-7f7js\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:04.135506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.135435    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:04.298929 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.298891    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:04.446349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.446328    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"]
Oct 30 13:21:04.448577 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:21:04.448547    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e6e1a7_832b_4a9f_8683_c66c5876c1f7.slice/crio-1c9e506fdce25c11fee8e45d218a78fd3bcd95d98877c2db30ec529912a9b6b0 WatchSource:0}: Error finding container 1c9e506fdce25c11fee8e45d218a78fd3bcd95d98877c2db30ec529912a9b6b0: Status 404 returned error can't find the container with id 1c9e506fdce25c11fee8e45d218a78fd3bcd95d98877c2db30ec529912a9b6b0
Oct 30 13:21:04.650194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:04.650170    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js" event={"ID":"75e6e1a7-832b-4a9f-8683-c66c5876c1f7","Type":"ContainerStarted","Data":"1c9e506fdce25c11fee8e45d218a78fd3bcd95d98877c2db30ec529912a9b6b0"}
Oct 30 13:21:05.658662 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:05.658584    5781 generic.go:356] "Generic (PLEG): container finished" podID="75e6e1a7-832b-4a9f-8683-c66c5876c1f7" containerID="89967fec7c56ee7c284f88de383da1f3db325333c830d0be45c15f14832e0647" exitCode=0
Oct 30 13:21:05.659022 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:05.658671    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js" event={"ID":"75e6e1a7-832b-4a9f-8683-c66c5876c1f7","Type":"ContainerDied","Data":"89967fec7c56ee7c284f88de383da1f3db325333c830d0be45c15f14832e0647"}
Oct 30 13:21:05.794017 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:05.793968    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:06.037768 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.037738    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"]
Oct 30 13:21:06.040776 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.040752    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1cf5c368-kfw42"]
Oct 30 13:21:06.069629 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.069600    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6" podUID="72ed25d8-f936-44c6-9003-79ff1d8da414" containerName="payload-pre-processing" containerID="cri-o://f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5" gracePeriod=25
Oct 30 13:21:06.808969 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.808944    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:06.840606 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.840574    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b9a4228-15a7-4d39-bc2f-767622117ce9" path="/var/lib/kubelet/pods/0b9a4228-15a7-4d39-bc2f-767622117ce9/volumes"
Oct 30 13:21:06.950550 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.950523    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-kube-api-access-bhmx9\" (UniqueName: \"kubernetes.io/projected/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-kube-api-access-bhmx9\") pod \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") "
Oct 30 13:21:06.950700 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.950574    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-openshift-ca\") pod \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") "
Oct 30 13:21:06.950746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.950717    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-xks-ca\") pod \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\" (UID: \"75e6e1a7-832b-4a9f-8683-c66c5876c1f7\") "
Oct 30 13:21:06.950933 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.950913    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-openshift-ca" pod "75e6e1a7-832b-4a9f-8683-c66c5876c1f7" (UID: "75e6e1a7-832b-4a9f-8683-c66c5876c1f7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:06.951132 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.951112    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:06.952651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.952629    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-kube-api-access-bhmx9" pod "75e6e1a7-832b-4a9f-8683-c66c5876c1f7" (UID: "75e6e1a7-832b-4a9f-8683-c66c5876c1f7"). InnerVolumeSpecName "kube-api-access-bhmx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:06.952750 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:06.952690    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-xks-ca" pod "75e6e1a7-832b-4a9f-8683-c66c5876c1f7" (UID: "75e6e1a7-832b-4a9f-8683-c66c5876c1f7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:07.052426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:07.052402    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:07.052426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:07.052424    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhmx9\" (UniqueName: \"kubernetes.io/projected/75e6e1a7-832b-4a9f-8683-c66c5876c1f7-kube-api-access-bhmx9\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:07.416509 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:07.416398    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:07.673794 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:07.673769    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"
Oct 30 13:21:07.673983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:07.673786    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js" event={"ID":"75e6e1a7-832b-4a9f-8683-c66c5876c1f7","Type":"ContainerDied","Data":"1c9e506fdce25c11fee8e45d218a78fd3bcd95d98877c2db30ec529912a9b6b0"}
Oct 30 13:21:07.673983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:07.673835    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c9e506fdce25c11fee8e45d218a78fd3bcd95d98877c2db30ec529912a9b6b0"
Oct 30 13:21:08.903331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:08.903265    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:10.430958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:10.430912    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:11.035759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:11.035720    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"]
Oct 30 13:21:11.046786 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:11.046756    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-21dae24e-6khm8"]
Oct 30 13:21:11.801815 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:11.801778    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:12.285782 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:12.285746    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:12.438243 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:12.438211    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:12.859280 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:12.859240    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4" path="/var/lib/kubelet/pods/01fb9ec6-241e-4c39-a1d5-c940ae7d1ed4/volumes"
Oct 30 13:21:12.974987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:12.974924    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:13.219020 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:13.218982    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:15.992317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:15.990497    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:16.023564 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:16.023506    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6" podUID="72ed25d8-f936-44c6-9003-79ff1d8da414" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:16.164143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:16.164108    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:16.375341 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:16.375268    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:16.537787 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:16.537754    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:17.985254 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:17.985205    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:19.528730 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:19.528699    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:19.731922 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:19.731889    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:20.631589 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.631554    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:20.795228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.795188    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:20.845195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.845148    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"]
Oct 30 13:21:20.845437 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.845362    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy" containerID="cri-o://0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318" gracePeriod=30
Oct 30 13:21:20.856434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.856410    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"]
Oct 30 13:21:20.856630 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.856609    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy" containerID="cri-o://1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0" gracePeriod=30
Oct 30 13:21:20.859088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.859064    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"]
Oct 30 13:21:20.859258 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.859237    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy" containerID="cri-o://db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea" gracePeriod=30
Oct 30 13:21:20.872056 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.872025    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"]
Oct 30 13:21:20.872597 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:20.872566    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy" containerID="cri-o://c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d" gracePeriod=30
Oct 30 13:21:21.007335 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:21.007279    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"]
Oct 30 13:21:21.007606 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:21.007576    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy" containerID="cri-o://04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646" gracePeriod=30
Oct 30 13:21:21.016181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:21.011525    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"]
Oct 30 13:21:21.016181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:21.011694    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy" containerID="cri-o://da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e" gracePeriod=30
Oct 30 13:21:21.016181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:21.015278    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"]
Oct 30 13:21:21.016181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:21.015488    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" containerID="cri-o://470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1" gracePeriod=30
Oct 30 13:21:21.125708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:21.125675    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:21.286072 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:21.285977    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:22.791950 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:22.791916    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:22.941721 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:22.941678    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:24.495365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:24.495331    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:21:24.495834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:24.495386    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:21:24.495834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:24.495751    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:24.654246 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:24.654200    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:25.038526 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:25.038492    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"]
Oct 30 13:21:25.051960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:25.051934    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9bf149d3-dzm8b"]
Oct 30 13:21:25.780618 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:25.780582    5781 patch_prober.go:29] interesting pod/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:21:25.780977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:25.780639    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:21:25.903168 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:25.903135    5781 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f0f44c_ee5a_4d0c_91c7_d1d55fd15c48.slice/crio-conmon-0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 13:21:25.903329 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:25.903170    5781 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9f0f44c_ee5a_4d0c_91c7_d1d55fd15c48.slice/crio-conmon-0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 13:21:25.986996 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:25.986883    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:26.002064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.002017    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6" podUID="72ed25d8-f936-44c6-9003-79ff1d8da414" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:26.305065 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.305038    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:21:26.308680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.308653    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:21:26.324479 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.324412    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:21:26.327530 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.327509    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:21:26.353969 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.353942    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:21:26.364055 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.364027    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:21:26.414942 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.414911    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-envoy\") pod \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") "
Oct 30 13:21:26.415099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.414958    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-istio-token\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-istio-token\") pod \"65918173-d800-4142-9f9a-fe8d32d5504d\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") "
Oct 30 13:21:26.415099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.414985    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-kube-api-access-r7kw6\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-kube-api-access-r7kw6\") pod \"8dd695c3-1426-4214-a757-704c120075cb\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") "
Oct 30 13:21:26.415099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415023    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-kube-api-access-f952l\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-kube-api-access-f952l\") pod \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") "
Oct 30 13:21:26.415099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415059    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/035c8706-673c-4624-8b95-57c566fba8db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/035c8706-673c-4624-8b95-57c566fba8db-istiod-ca-cert\") pod \"035c8706-673c-4624-8b95-57c566fba8db\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") "
Oct 30 13:21:26.415099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415085    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8dd695c3-1426-4214-a757-704c120075cb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd695c3-1426-4214-a757-704c120075cb-istio-podinfo\") pod \"8dd695c3-1426-4214-a757-704c120075cb\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") "
Oct 30 13:21:26.415407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415114    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-envoy\") pod \"035c8706-673c-4624-8b95-57c566fba8db\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") "
Oct 30 13:21:26.415407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415141    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-data\") pod \"035c8706-673c-4624-8b95-57c566fba8db\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") "
Oct 30 13:21:26.415407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415179    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-credential-socket\") pod \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") "
Oct 30 13:21:26.415407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415209    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-socket\") pod \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") "
Oct 30 13:21:26.415407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415244    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-istio-token\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-istio-token\") pod \"8dd695c3-1426-4214-a757-704c120075cb\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") "
Oct 30 13:21:26.415407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415279    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-certs\") pod \"035c8706-673c-4624-8b95-57c566fba8db\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") "
Oct 30 13:21:26.415407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415356    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istiod-ca-cert\") pod \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") "
Oct 30 13:21:26.415407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415386    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-data\") pod \"fc834143-970e-4e23-8140-fc53aec8ea54\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415415    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-envoy\") pod \"8dd695c3-1426-4214-a757-704c120075cb\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415442    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/65918173-d800-4142-9f9a-fe8d32d5504d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65918173-d800-4142-9f9a-fe8d32d5504d-istio-podinfo\") pod \"65918173-d800-4142-9f9a-fe8d32d5504d\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415468    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-kube-api-access-8zr9h\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-kube-api-access-8zr9h\") pod \"fc834143-970e-4e23-8140-fc53aec8ea54\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415501    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-envoy\") pod \"65918173-d800-4142-9f9a-fe8d32d5504d\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415535    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-certs\") pod \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415589    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-credential-socket\") pod \"65918173-d800-4142-9f9a-fe8d32d5504d\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415616    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-data\") pod \"65918173-d800-4142-9f9a-fe8d32d5504d\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415640    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-envoy\") pod \"fc834143-970e-4e23-8140-fc53aec8ea54\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415657    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-certs\") pod \"65918173-d800-4142-9f9a-fe8d32d5504d\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415687    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-kube-api-access-6rcrt\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-kube-api-access-6rcrt\") pod \"65918173-d800-4142-9f9a-fe8d32d5504d\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415721    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-socket\") pod \"fc834143-970e-4e23-8140-fc53aec8ea54\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") "
Oct 30 13:21:26.415743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415741    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-data\") pod \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415764    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-data\") pod \"8dd695c3-1426-4214-a757-704c120075cb\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415788    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fc834143-970e-4e23-8140-fc53aec8ea54-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc834143-970e-4e23-8140-fc53aec8ea54-istio-podinfo\") pod \"fc834143-970e-4e23-8140-fc53aec8ea54\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415819    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-socket\") pod \"035c8706-673c-4624-8b95-57c566fba8db\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415838    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-socket\") pod \"65918173-d800-4142-9f9a-fe8d32d5504d\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415877    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/72ed25d8-f936-44c6-9003-79ff1d8da414-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72ed25d8-f936-44c6-9003-79ff1d8da414-plugins-config-volume\") pod \"72ed25d8-f936-44c6-9003-79ff1d8da414\" (UID: \"72ed25d8-f936-44c6-9003-79ff1d8da414\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415909    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-istio-token\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-istio-token\") pod \"035c8706-673c-4624-8b95-57c566fba8db\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415943    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-istio-token\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-istio-token\") pod \"fc834143-970e-4e23-8140-fc53aec8ea54\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.415999    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-podinfo\") pod \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416025    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-credential-socket\") pod \"8dd695c3-1426-4214-a757-704c120075cb\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416068    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-kube-api-access-6cgxr\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-kube-api-access-6cgxr\") pod \"035c8706-673c-4624-8b95-57c566fba8db\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416095    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-credential-socket\") pod \"035c8706-673c-4624-8b95-57c566fba8db\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416121    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65918173-d800-4142-9f9a-fe8d32d5504d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65918173-d800-4142-9f9a-fe8d32d5504d-istiod-ca-cert\") pod \"65918173-d800-4142-9f9a-fe8d32d5504d\" (UID: \"65918173-d800-4142-9f9a-fe8d32d5504d\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416162    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-certs\") pod \"fc834143-970e-4e23-8140-fc53aec8ea54\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416195    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc834143-970e-4e23-8140-fc53aec8ea54-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc834143-970e-4e23-8140-fc53aec8ea54-istiod-ca-cert\") pod \"fc834143-970e-4e23-8140-fc53aec8ea54\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") "
Oct 30 13:21:26.416234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416234    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-socket\") pod \"8dd695c3-1426-4214-a757-704c120075cb\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") "
Oct 30 13:21:26.416910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416269    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-credential-socket\") pod \"fc834143-970e-4e23-8140-fc53aec8ea54\" (UID: \"fc834143-970e-4e23-8140-fc53aec8ea54\") "
Oct 30 13:21:26.416910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416345    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-certs\") pod \"8dd695c3-1426-4214-a757-704c120075cb\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") "
Oct 30 13:21:26.416910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416371    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8dd695c3-1426-4214-a757-704c120075cb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd695c3-1426-4214-a757-704c120075cb-istiod-ca-cert\") pod \"8dd695c3-1426-4214-a757-704c120075cb\" (UID: \"8dd695c3-1426-4214-a757-704c120075cb\") "
Oct 30 13:21:26.416910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416407    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/72ed25d8-f936-44c6-9003-79ff1d8da414-kube-api-access-hzlc9\" (UniqueName: \"kubernetes.io/projected/72ed25d8-f936-44c6-9003-79ff1d8da414-kube-api-access-hzlc9\") pod \"72ed25d8-f936-44c6-9003-79ff1d8da414\" (UID: \"72ed25d8-f936-44c6-9003-79ff1d8da414\") "
Oct 30 13:21:26.416910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416442    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/035c8706-673c-4624-8b95-57c566fba8db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/035c8706-673c-4624-8b95-57c566fba8db-istio-podinfo\") pod \"035c8706-673c-4624-8b95-57c566fba8db\" (UID: \"035c8706-673c-4624-8b95-57c566fba8db\") "
Oct 30 13:21:26.416910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.416466    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-token\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-token\") pod \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\" (UID: \"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48\") "
Oct 30 13:21:26.417500 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.417477    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-certs" pod "65918173-d800-4142-9f9a-fe8d32d5504d" (UID: "65918173-d800-4142-9f9a-fe8d32d5504d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.417627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.417602    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-socket" pod "035c8706-673c-4624-8b95-57c566fba8db" (UID: "035c8706-673c-4624-8b95-57c566fba8db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.419748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.418350    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-socket" pod "8dd695c3-1426-4214-a757-704c120075cb" (UID: "8dd695c3-1426-4214-a757-704c120075cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.419748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.418364    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-certs" pod "035c8706-673c-4624-8b95-57c566fba8db" (UID: "035c8706-673c-4624-8b95-57c566fba8db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.419748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.418564    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-data" pod "8dd695c3-1426-4214-a757-704c120075cb" (UID: "8dd695c3-1426-4214-a757-704c120075cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.419748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.418885    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65918173-d800-4142-9f9a-fe8d32d5504d-istiod-ca-cert" pod "65918173-d800-4142-9f9a-fe8d32d5504d" (UID: "65918173-d800-4142-9f9a-fe8d32d5504d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.419748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.418888    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istiod-ca-cert" pod "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" (UID: "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.419748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.419181    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-certs" pod "fc834143-970e-4e23-8140-fc53aec8ea54" (UID: "fc834143-970e-4e23-8140-fc53aec8ea54"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.419748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.419567    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc834143-970e-4e23-8140-fc53aec8ea54-istiod-ca-cert" pod "fc834143-970e-4e23-8140-fc53aec8ea54" (UID: "fc834143-970e-4e23-8140-fc53aec8ea54"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.419748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.419639    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-credential-socket" pod "fc834143-970e-4e23-8140-fc53aec8ea54" (UID: "fc834143-970e-4e23-8140-fc53aec8ea54"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.419748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.419701    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-credential-socket" pod "035c8706-673c-4624-8b95-57c566fba8db" (UID: "035c8706-673c-4624-8b95-57c566fba8db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.421760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.419791    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-token" pod "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" (UID: "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.421760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.419814    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-certs" pod "8dd695c3-1426-4214-a757-704c120075cb" (UID: "8dd695c3-1426-4214-a757-704c120075cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.421760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.419854    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-data" pod "fc834143-970e-4e23-8140-fc53aec8ea54" (UID: "fc834143-970e-4e23-8140-fc53aec8ea54"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.421760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.420080    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-socket" pod "fc834143-970e-4e23-8140-fc53aec8ea54" (UID: "fc834143-970e-4e23-8140-fc53aec8ea54"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.421760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.420110    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd695c3-1426-4214-a757-704c120075cb-istiod-ca-cert" pod "8dd695c3-1426-4214-a757-704c120075cb" (UID: "8dd695c3-1426-4214-a757-704c120075cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.421760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.420274    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-data" pod "035c8706-673c-4624-8b95-57c566fba8db" (UID: "035c8706-673c-4624-8b95-57c566fba8db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.421760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.420620    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/035c8706-673c-4624-8b95-57c566fba8db-istiod-ca-cert" pod "035c8706-673c-4624-8b95-57c566fba8db" (UID: "035c8706-673c-4624-8b95-57c566fba8db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.422456 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.422371    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-data" pod "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" (UID: "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.423000    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8dd695c3-1426-4214-a757-704c120075cb-istio-podinfo" pod "8dd695c3-1426-4214-a757-704c120075cb" (UID: "8dd695c3-1426-4214-a757-704c120075cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.424218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.423130    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-kube-api-access-r7kw6" pod "8dd695c3-1426-4214-a757-704c120075cb" (UID: "8dd695c3-1426-4214-a757-704c120075cb"). InnerVolumeSpecName "kube-api-access-r7kw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.424218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.423185    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-envoy" pod "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" (UID: "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.423378    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-certs" pod "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" (UID: "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.423466    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-credential-socket" pod "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" (UID: "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.423625    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-socket" pod "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" (UID: "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.423799    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-credential-socket" pod "65918173-d800-4142-9f9a-fe8d32d5504d" (UID: "65918173-d800-4142-9f9a-fe8d32d5504d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.424218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.423951    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-data" pod "65918173-d800-4142-9f9a-fe8d32d5504d" (UID: "65918173-d800-4142-9f9a-fe8d32d5504d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.425492 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.424828    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-socket" pod "65918173-d800-4142-9f9a-fe8d32d5504d" (UID: "65918173-d800-4142-9f9a-fe8d32d5504d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.425492 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.425120    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72ed25d8-f936-44c6-9003-79ff1d8da414-plugins-config-volume" pod "72ed25d8-f936-44c6-9003-79ff1d8da414" (UID: "72ed25d8-f936-44c6-9003-79ff1d8da414"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.425492 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.425210    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-envoy" pod "035c8706-673c-4624-8b95-57c566fba8db" (UID: "035c8706-673c-4624-8b95-57c566fba8db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.425492 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.425363    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-envoy" pod "65918173-d800-4142-9f9a-fe8d32d5504d" (UID: "65918173-d800-4142-9f9a-fe8d32d5504d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.425821 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.425800    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-istio-token" pod "fc834143-970e-4e23-8140-fc53aec8ea54" (UID: "fc834143-970e-4e23-8140-fc53aec8ea54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.426107 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.426072    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-credential-socket" pod "8dd695c3-1426-4214-a757-704c120075cb" (UID: "8dd695c3-1426-4214-a757-704c120075cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.426280 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.426239    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-istio-token" pod "035c8706-673c-4624-8b95-57c566fba8db" (UID: "035c8706-673c-4624-8b95-57c566fba8db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.426723 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.426688    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-envoy" pod "fc834143-970e-4e23-8140-fc53aec8ea54" (UID: "fc834143-970e-4e23-8140-fc53aec8ea54"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.427050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.426829    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-podinfo" pod "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" (UID: "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.428097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.428071    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-envoy" pod "8dd695c3-1426-4214-a757-704c120075cb" (UID: "8dd695c3-1426-4214-a757-704c120075cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.428495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.428445    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/035c8706-673c-4624-8b95-57c566fba8db-istio-podinfo" pod "035c8706-673c-4624-8b95-57c566fba8db" (UID: "035c8706-673c-4624-8b95-57c566fba8db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.428495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.428465    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-istio-token" pod "8dd695c3-1426-4214-a757-704c120075cb" (UID: "8dd695c3-1426-4214-a757-704c120075cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.430667 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.430643    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72ed25d8-f936-44c6-9003-79ff1d8da414-kube-api-access-hzlc9" pod "72ed25d8-f936-44c6-9003-79ff1d8da414" (UID: "72ed25d8-f936-44c6-9003-79ff1d8da414"). InnerVolumeSpecName "kube-api-access-hzlc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.430773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.430667    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc834143-970e-4e23-8140-fc53aec8ea54-istio-podinfo" pod "fc834143-970e-4e23-8140-fc53aec8ea54" (UID: "fc834143-970e-4e23-8140-fc53aec8ea54"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.430773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.430763    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-kube-api-access-6cgxr" pod "035c8706-673c-4624-8b95-57c566fba8db" (UID: "035c8706-673c-4624-8b95-57c566fba8db"). InnerVolumeSpecName "kube-api-access-6cgxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.430879 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.430801    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-istio-token" pod "65918173-d800-4142-9f9a-fe8d32d5504d" (UID: "65918173-d800-4142-9f9a-fe8d32d5504d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.430879 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.430828    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-kube-api-access-6rcrt" pod "65918173-d800-4142-9f9a-fe8d32d5504d" (UID: "65918173-d800-4142-9f9a-fe8d32d5504d"). InnerVolumeSpecName "kube-api-access-6rcrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.430879 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.430848    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-kube-api-access-8zr9h" pod "fc834143-970e-4e23-8140-fc53aec8ea54" (UID: "fc834143-970e-4e23-8140-fc53aec8ea54"). InnerVolumeSpecName "kube-api-access-8zr9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.431810 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.431784    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-kube-api-access-f952l" pod "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" (UID: "e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48"). InnerVolumeSpecName "kube-api-access-f952l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.432101 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.432078    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65918173-d800-4142-9f9a-fe8d32d5504d-istio-podinfo" pod "65918173-d800-4142-9f9a-fe8d32d5504d" (UID: "65918173-d800-4142-9f9a-fe8d32d5504d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.466944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.466920    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:26.518331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518276    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cgxr\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-kube-api-access-6cgxr\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518330    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518340    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65918173-d800-4142-9f9a-fe8d32d5504d-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518349    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518357    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc834143-970e-4e23-8140-fc53aec8ea54-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518366    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518378    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518387    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518394    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd695c3-1426-4214-a757-704c120075cb-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518402    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzlc9\" (UniqueName: \"kubernetes.io/projected/72ed25d8-f936-44c6-9003-79ff1d8da414-kube-api-access-hzlc9\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518409    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/035c8706-673c-4624-8b95-57c566fba8db-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518424    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518433    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518439    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518450    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7kw6\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-kube-api-access-r7kw6\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518461    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f952l\" (UniqueName: \"kubernetes.io/projected/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-kube-api-access-f952l\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518469    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/035c8706-673c-4624-8b95-57c566fba8db-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518476    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd695c3-1426-4214-a757-704c120075cb-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518484    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518491    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518498    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518506    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518513    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dd695c3-1426-4214-a757-704c120075cb-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518524    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.518525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518534    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518541    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518550    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518557    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65918173-d800-4142-9f9a-fe8d32d5504d-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518564    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zr9h\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-kube-api-access-8zr9h\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518573    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518581    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518588    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518595    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518608    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518616    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518623    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rcrt\" (UniqueName: \"kubernetes.io/projected/65918173-d800-4142-9f9a-fe8d32d5504d-kube-api-access-6rcrt\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518630    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc834143-970e-4e23-8140-fc53aec8ea54-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518638    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518646    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518653    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc834143-970e-4e23-8140-fc53aec8ea54-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518660    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/035c8706-673c-4624-8b95-57c566fba8db-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518668    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65918173-d800-4142-9f9a-fe8d32d5504d-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518679    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/72ed25d8-f936-44c6-9003-79ff1d8da414-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518691    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/035c8706-673c-4624-8b95-57c566fba8db-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518699    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc834143-970e-4e23-8140-fc53aec8ea54-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518706    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.519143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.518714    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd695c3-1426-4214-a757-704c120075cb-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.689991 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.689963    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:21:26.732797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.732770    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:21:26.823596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823566    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-data\") pod \"5eb08489-1221-4ab8-9b10-df2003f17b23\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") "
Oct 30 13:21:26.823596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823602    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d38948d-2d60-403c-91d0-0b5f1a534e60-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d38948d-2d60-403c-91d0-0b5f1a534e60-istiod-ca-cert\") pod \"2d38948d-2d60-403c-91d0-0b5f1a534e60\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823630    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-token\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-token\") pod \"2d38948d-2d60-403c-91d0-0b5f1a534e60\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823665    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-envoy\") pod \"2d38948d-2d60-403c-91d0-0b5f1a534e60\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823689    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-kube-api-access-x2n2c\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-kube-api-access-x2n2c\") pod \"2d38948d-2d60-403c-91d0-0b5f1a534e60\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823722    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-token\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-token\") pod \"5eb08489-1221-4ab8-9b10-df2003f17b23\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823762    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5eb08489-1221-4ab8-9b10-df2003f17b23-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb08489-1221-4ab8-9b10-df2003f17b23-istiod-ca-cert\") pod \"5eb08489-1221-4ab8-9b10-df2003f17b23\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823802    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-podinfo\") pod \"2d38948d-2d60-403c-91d0-0b5f1a534e60\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823842    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-credential-socket\") pod \"5eb08489-1221-4ab8-9b10-df2003f17b23\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823867    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-data\") pod \"2d38948d-2d60-403c-91d0-0b5f1a534e60\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823900    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-data" pod "5eb08489-1221-4ab8-9b10-df2003f17b23" (UID: "5eb08489-1221-4ab8-9b10-df2003f17b23"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823914    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-envoy\") pod \"5eb08489-1221-4ab8-9b10-df2003f17b23\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823962    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-kube-api-access-7qwbr\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-kube-api-access-7qwbr\") pod \"5eb08489-1221-4ab8-9b10-df2003f17b23\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.823987    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d38948d-2d60-403c-91d0-0b5f1a534e60-istiod-ca-cert" pod "2d38948d-2d60-403c-91d0-0b5f1a534e60" (UID: "2d38948d-2d60-403c-91d0-0b5f1a534e60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824000    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-podinfo\") pod \"5eb08489-1221-4ab8-9b10-df2003f17b23\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") "
Oct 30 13:21:26.824084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824055    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-certs\") pod \"5eb08489-1221-4ab8-9b10-df2003f17b23\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") "
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824107    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-socket\") pod \"5eb08489-1221-4ab8-9b10-df2003f17b23\" (UID: \"5eb08489-1221-4ab8-9b10-df2003f17b23\") "
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824159    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-credential-socket\") pod \"2d38948d-2d60-403c-91d0-0b5f1a534e60\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") "
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824220    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-certs\") pod \"2d38948d-2d60-403c-91d0-0b5f1a534e60\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") "
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824277    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-socket\") pod \"2d38948d-2d60-403c-91d0-0b5f1a534e60\" (UID: \"2d38948d-2d60-403c-91d0-0b5f1a534e60\") "
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824336    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-data" pod "2d38948d-2d60-403c-91d0-0b5f1a534e60" (UID: "2d38948d-2d60-403c-91d0-0b5f1a534e60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824535    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-credential-socket" pod "5eb08489-1221-4ab8-9b10-df2003f17b23" (UID: "5eb08489-1221-4ab8-9b10-df2003f17b23"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824469    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb08489-1221-4ab8-9b10-df2003f17b23-istiod-ca-cert" pod "5eb08489-1221-4ab8-9b10-df2003f17b23" (UID: "5eb08489-1221-4ab8-9b10-df2003f17b23"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824733    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824733    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-socket" pod "2d38948d-2d60-403c-91d0-0b5f1a534e60" (UID: "2d38948d-2d60-403c-91d0-0b5f1a534e60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824751    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d38948d-2d60-403c-91d0-0b5f1a534e60-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824764    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eb08489-1221-4ab8-9b10-df2003f17b23-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824765    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-certs" pod "5eb08489-1221-4ab8-9b10-df2003f17b23" (UID: "5eb08489-1221-4ab8-9b10-df2003f17b23"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824779    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.824820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824792    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.825526 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824921    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-socket" pod "5eb08489-1221-4ab8-9b10-df2003f17b23" (UID: "5eb08489-1221-4ab8-9b10-df2003f17b23"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.825526 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.824990    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-credential-socket" pod "2d38948d-2d60-403c-91d0-0b5f1a534e60" (UID: "2d38948d-2d60-403c-91d0-0b5f1a534e60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.825526 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.825112    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-certs" pod "2d38948d-2d60-403c-91d0-0b5f1a534e60" (UID: "2d38948d-2d60-403c-91d0-0b5f1a534e60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.826562 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.826525    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-podinfo" pod "2d38948d-2d60-403c-91d0-0b5f1a534e60" (UID: "2d38948d-2d60-403c-91d0-0b5f1a534e60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.826921 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.826888    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-envoy" pod "5eb08489-1221-4ab8-9b10-df2003f17b23" (UID: "5eb08489-1221-4ab8-9b10-df2003f17b23"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.827102 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.827084    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-envoy" pod "2d38948d-2d60-403c-91d0-0b5f1a534e60" (UID: "2d38948d-2d60-403c-91d0-0b5f1a534e60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:26.827178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.827097    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-token" pod "2d38948d-2d60-403c-91d0-0b5f1a534e60" (UID: "2d38948d-2d60-403c-91d0-0b5f1a534e60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.827178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.827129    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-kube-api-access-7qwbr" pod "5eb08489-1221-4ab8-9b10-df2003f17b23" (UID: "5eb08489-1221-4ab8-9b10-df2003f17b23"). InnerVolumeSpecName "kube-api-access-7qwbr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.827178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.827139    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-kube-api-access-x2n2c" pod "2d38948d-2d60-403c-91d0-0b5f1a534e60" (UID: "2d38948d-2d60-403c-91d0-0b5f1a534e60"). InnerVolumeSpecName "kube-api-access-x2n2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.827377 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.827241    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-podinfo" pod "5eb08489-1221-4ab8-9b10-df2003f17b23" (UID: "5eb08489-1221-4ab8-9b10-df2003f17b23"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:26.828249 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.828233    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-token" pod "5eb08489-1221-4ab8-9b10-df2003f17b23" (UID: "5eb08489-1221-4ab8-9b10-df2003f17b23"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:26.835603 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.835583    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:26.842047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.842021    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ddd7f33-80f2-4517-9673-1bf6e028e6b7" path="/var/lib/kubelet/pods/5ddd7f33-80f2-4517-9673-1bf6e028e6b7/volumes"
Oct 30 13:21:26.855044 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.854902    5781 generic.go:356] "Generic (PLEG): container finished" podID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerID="04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646" exitCode=0
Oct 30 13:21:26.855044 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.854949    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"
Oct 30 13:21:26.855044 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.855024    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" event={"ID":"5eb08489-1221-4ab8-9b10-df2003f17b23","Type":"ContainerDied","Data":"04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646"}
Oct 30 13:21:26.855197 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.855068    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld" event={"ID":"5eb08489-1221-4ab8-9b10-df2003f17b23","Type":"ContainerDied","Data":"3c2dcfeb56f974febfd4526eadccc941af807d2a8093224695f295ea7a0759ab"}
Oct 30 13:21:26.855197 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.855097    5781 scope.go:122] "RemoveContainer" containerID="04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646"
Oct 30 13:21:26.859267 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.859242    5781 generic.go:356] "Generic (PLEG): container finished" podID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerID="470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1" exitCode=0
Oct 30 13:21:26.859426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.859397    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" event={"ID":"2d38948d-2d60-403c-91d0-0b5f1a534e60","Type":"ContainerDied","Data":"470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1"}
Oct 30 13:21:26.859503 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.859486    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk" event={"ID":"2d38948d-2d60-403c-91d0-0b5f1a534e60","Type":"ContainerDied","Data":"3682dd3dd9cc9c84cc2cf70e1b6786595c7ef91b5398ae8f1d80aef8bdaebdc4"}
Oct 30 13:21:26.859627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.859610    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"
Oct 30 13:21:26.869680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.869536    5781 generic.go:356] "Generic (PLEG): container finished" podID="65918173-d800-4142-9f9a-fe8d32d5504d" containerID="db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea" exitCode=0
Oct 30 13:21:26.869680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.869621    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"
Oct 30 13:21:26.869680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.869631    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" event={"ID":"65918173-d800-4142-9f9a-fe8d32d5504d","Type":"ContainerDied","Data":"db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea"}
Oct 30 13:21:26.869960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.869683    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh" event={"ID":"65918173-d800-4142-9f9a-fe8d32d5504d","Type":"ContainerDied","Data":"3d4eaa53a5b0551e707e07c84e2510cd7aec2b82df5127618689eab181d4bd68"}
Oct 30 13:21:26.872018 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.871895    5781 generic.go:356] "Generic (PLEG): container finished" podID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerID="0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318" exitCode=0
Oct 30 13:21:26.872018 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.871984    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"
Oct 30 13:21:26.872205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.872013    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" event={"ID":"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48","Type":"ContainerDied","Data":"0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318"}
Oct 30 13:21:26.872205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.872066    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8" event={"ID":"e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48","Type":"ContainerDied","Data":"b7a98787e1c22cb948b261a32a29c96f0ad4a28960d3345c25d0f352a5f8b7a6"}
Oct 30 13:21:26.874202 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.874189    5781 generic.go:356] "Generic (PLEG): container finished" podID="035c8706-673c-4624-8b95-57c566fba8db" containerID="1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0" exitCode=0
Oct 30 13:21:26.874275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.874255    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"
Oct 30 13:21:26.874396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.874263    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" event={"ID":"035c8706-673c-4624-8b95-57c566fba8db","Type":"ContainerDied","Data":"1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0"}
Oct 30 13:21:26.874463 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.874399    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l" event={"ID":"035c8706-673c-4624-8b95-57c566fba8db","Type":"ContainerDied","Data":"8fbea125afc3bc4e6a9a28e90d13950cf4fe59318b092ef13d7b4ef1a7dd278a"}
Oct 30 13:21:26.876506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.876481    5781 generic.go:356] "Generic (PLEG): container finished" podID="fc834143-970e-4e23-8140-fc53aec8ea54" containerID="c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d" exitCode=0
Oct 30 13:21:26.876605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.876536    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"
Oct 30 13:21:26.876680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.876645    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" event={"ID":"fc834143-970e-4e23-8140-fc53aec8ea54","Type":"ContainerDied","Data":"c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d"}
Oct 30 13:21:26.877076 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.877031    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c" event={"ID":"fc834143-970e-4e23-8140-fc53aec8ea54","Type":"ContainerDied","Data":"36d0234ded1abe36f890bcc212234b7d19085b4d530482f20d58a7f529f236a0"}
Oct 30 13:21:26.884073 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.884053    5781 generic.go:356] "Generic (PLEG): container finished" podID="72ed25d8-f936-44c6-9003-79ff1d8da414" containerID="f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5" exitCode=0
Oct 30 13:21:26.884155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.884105    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"
Oct 30 13:21:26.884155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.884115    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6" event={"ID":"72ed25d8-f936-44c6-9003-79ff1d8da414","Type":"ContainerDied","Data":"f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5"}
Oct 30 13:21:26.884155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.884149    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6" event={"ID":"72ed25d8-f936-44c6-9003-79ff1d8da414","Type":"ContainerDied","Data":"3de367902f89dcb264a660a1952bbde4a2b82a1d05d782c84b1a71fd2c7ef1b6"}
Oct 30 13:21:26.885349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.885331    5781 generic.go:356] "Generic (PLEG): container finished" podID="8dd695c3-1426-4214-a757-704c120075cb" containerID="da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e" exitCode=0
Oct 30 13:21:26.885435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.885371    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" event={"ID":"8dd695c3-1426-4214-a757-704c120075cb","Type":"ContainerDied","Data":"da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e"}
Oct 30 13:21:26.885435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.885395    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw" event={"ID":"8dd695c3-1426-4214-a757-704c120075cb","Type":"ContainerDied","Data":"a7da27d8c370ade327ef47bd18b891c836366c9a197b7719920ca3a2b3bd6da5"}
Oct 30 13:21:26.885516 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.885445    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"
Oct 30 13:21:26.919182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.919162    5781 scope.go:122] "RemoveContainer" containerID="3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a"
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926308    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926337    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926363    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2n2c\" (UniqueName: \"kubernetes.io/projected/2d38948d-2d60-403c-91d0-0b5f1a534e60-kube-api-access-x2n2c\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926372    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926381    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d38948d-2d60-403c-91d0-0b5f1a534e60-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926388    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926397    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qwbr\" (UniqueName: \"kubernetes.io/projected/5eb08489-1221-4ab8-9b10-df2003f17b23-kube-api-access-7qwbr\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926405    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eb08489-1221-4ab8-9b10-df2003f17b23-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926422    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926431    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eb08489-1221-4ab8-9b10-df2003f17b23-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926439    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926446    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.926469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.926455    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d38948d-2d60-403c-91d0-0b5f1a534e60-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:26.930980 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.930957    5781 scope.go:122] "RemoveContainer" containerID="04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646"
Oct 30 13:21:26.931299 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:26.931266    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646\": container with ID starting with 04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646 not found: ID does not exist" containerID="04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646"
Oct 30 13:21:26.931381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.931323    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646"} err="failed to get container status \"04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646\": rpc error: code = NotFound desc = could not find container \"04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646\": container with ID starting with 04d082925c462f08af0f63a1e7c7e6d79e7377ba0d4856efee2dddfebdea3646 not found: ID does not exist"
Oct 30 13:21:26.931381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.931348    5781 scope.go:122] "RemoveContainer" containerID="3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a"
Oct 30 13:21:26.931664 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:26.931644    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a\": container with ID starting with 3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a not found: ID does not exist" containerID="3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a"
Oct 30 13:21:26.931757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.931667    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a"} err="failed to get container status \"3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a\": rpc error: code = NotFound desc = could not find container \"3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a\": container with ID starting with 3ec8c30019a400f9697712ff0073143be5f77046bb8164e95e92373a13ffa68a not found: ID does not exist"
Oct 30 13:21:26.931757 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.931680    5781 scope.go:122] "RemoveContainer" containerID="470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1"
Oct 30 13:21:26.940681 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.940662    5781 scope.go:122] "RemoveContainer" containerID="470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1"
Oct 30 13:21:26.940933 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:26.940915    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1\": container with ID starting with 470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1 not found: ID does not exist" containerID="470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1"
Oct 30 13:21:26.941005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.940943    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1"} err="failed to get container status \"470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1\": rpc error: code = NotFound desc = could not find container \"470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1\": container with ID starting with 470c35b87a2c7175bc3292011141981f67f6ca0eb13d96a4bd176226fb23cde1 not found: ID does not exist"
Oct 30 13:21:26.941005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.940965    5781 scope.go:122] "RemoveContainer" containerID="db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea"
Oct 30 13:21:26.949039 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.949020    5781 scope.go:122] "RemoveContainer" containerID="db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea"
Oct 30 13:21:26.949300 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:26.949273    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea\": container with ID starting with db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea not found: ID does not exist" containerID="db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea"
Oct 30 13:21:26.949367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.949304    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea"} err="failed to get container status \"db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea\": rpc error: code = NotFound desc = could not find container \"db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea\": container with ID starting with db60a9e844598ce19be8763cbe5d16fd31d608baa9cac273c8fb57e45b9aa0ea not found: ID does not exist"
Oct 30 13:21:26.949367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.949321    5781 scope.go:122] "RemoveContainer" containerID="0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318"
Oct 30 13:21:26.957645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.957630    5781 scope.go:122] "RemoveContainer" containerID="0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318"
Oct 30 13:21:26.957907 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:26.957890    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318\": container with ID starting with 0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318 not found: ID does not exist" containerID="0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318"
Oct 30 13:21:26.957952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.957919    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318"} err="failed to get container status \"0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318\": rpc error: code = NotFound desc = could not find container \"0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318\": container with ID starting with 0f5d4fb9aeb8edfafb1c47801a485bd67807c9c4b5b7c83717b84243cc927318 not found: ID does not exist"
Oct 30 13:21:26.957952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.957933    5781 scope.go:122] "RemoveContainer" containerID="1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0"
Oct 30 13:21:26.965777 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.965763    5781 scope.go:122] "RemoveContainer" containerID="1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0"
Oct 30 13:21:26.966019 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:26.965993    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0\": container with ID starting with 1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0 not found: ID does not exist" containerID="1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0"
Oct 30 13:21:26.966066 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.966015    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0"} err="failed to get container status \"1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0\": rpc error: code = NotFound desc = could not find container \"1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0\": container with ID starting with 1ed7abfd0d420eb9b338adc069ec007bb3add58c3fee612bf0c18ba92e225da0 not found: ID does not exist"
Oct 30 13:21:26.966066 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.966030    5781 scope.go:122] "RemoveContainer" containerID="c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d"
Oct 30 13:21:26.977091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.977066    5781 scope.go:122] "RemoveContainer" containerID="c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d"
Oct 30 13:21:26.977696 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:26.977671    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d\": container with ID starting with c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d not found: ID does not exist" containerID="c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d"
Oct 30 13:21:26.977762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.977699    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d"} err="failed to get container status \"c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d\": rpc error: code = NotFound desc = could not find container \"c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d\": container with ID starting with c7ed60b70c1abfa3e68180ea65455ba940a2fffe39e3ad790dc890332040a27d not found: ID does not exist"
Oct 30 13:21:26.977762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.977715    5781 scope.go:122] "RemoveContainer" containerID="f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5"
Oct 30 13:21:26.986113 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.985950    5781 scope.go:122] "RemoveContainer" containerID="f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5"
Oct 30 13:21:26.986230 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:26.986213    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5\": container with ID starting with f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5 not found: ID does not exist" containerID="f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5"
Oct 30 13:21:26.986270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.986235    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5"} err="failed to get container status \"f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5\": rpc error: code = NotFound desc = could not find container \"f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5\": container with ID starting with f1af1823751fcb30ffd7e06df0ff41331b436ed1d93dd466c2491a5b8e3ec5d5 not found: ID does not exist"
Oct 30 13:21:26.986270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.986248    5781 scope.go:122] "RemoveContainer" containerID="da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e"
Oct 30 13:21:26.994185 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.994167    5781 scope.go:122] "RemoveContainer" containerID="da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e"
Oct 30 13:21:26.994500 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:26.994480    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e\": container with ID starting with da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e not found: ID does not exist" containerID="da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e"
Oct 30 13:21:26.994577 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:26.994503    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e"} err="failed to get container status \"da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e\": rpc error: code = NotFound desc = could not find container \"da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e\": container with ID starting with da853ab2abace954572ce2819460f058e565e39c9b15bab0fca499634ff20f4e not found: ID does not exist"
Oct 30 13:21:27.025947 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.025921    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"]
Oct 30 13:21:27.029610 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.029591    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-6fzdk"]
Oct 30 13:21:27.036635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.036616    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:27.079960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.079940    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"]
Oct 30 13:21:27.084669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.084650    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-vnckw"]
Oct 30 13:21:27.131353 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.131275    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"]
Oct 30 13:21:27.139359 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.139339    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6797c"]
Oct 30 13:21:27.200995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.200964    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"]
Oct 30 13:21:27.209132 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.209108    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-f76c09-openshift-default-65d5845b67-n9pld"]
Oct 30 13:21:27.245746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.245711    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"]
Oct 30 13:21:27.252412 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.252381    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zt4gh"]
Oct 30 13:21:27.286471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.286437    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"]
Oct 30 13:21:27.292439 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.292414    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-5r99l"]
Oct 30 13:21:27.331809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.331754    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"]
Oct 30 13:21:27.335429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.335404    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-88f6a4-59bbc879dc-gz9q6"]
Oct 30 13:21:27.381660 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.381575    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"]
Oct 30 13:21:27.384342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.384316    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xzzm8"]
Oct 30 13:21:27.440621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.440579    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:27.602929 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:27.602880    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:28.049336 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:28.049306    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:28.840763 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:28.840732    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="035c8706-673c-4624-8b95-57c566fba8db" path="/var/lib/kubelet/pods/035c8706-673c-4624-8b95-57c566fba8db/volumes"
Oct 30 13:21:28.841152 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:28.841139    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" path="/var/lib/kubelet/pods/2d38948d-2d60-403c-91d0-0b5f1a534e60/volumes"
Oct 30 13:21:28.841542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:28.841530    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" path="/var/lib/kubelet/pods/5eb08489-1221-4ab8-9b10-df2003f17b23/volumes"
Oct 30 13:21:28.842247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:28.842230    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" path="/var/lib/kubelet/pods/65918173-d800-4142-9f9a-fe8d32d5504d/volumes"
Oct 30 13:21:28.842598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:28.842585    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="72ed25d8-f936-44c6-9003-79ff1d8da414" path="/var/lib/kubelet/pods/72ed25d8-f936-44c6-9003-79ff1d8da414/volumes"
Oct 30 13:21:28.842974 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:28.842959    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8dd695c3-1426-4214-a757-704c120075cb" path="/var/lib/kubelet/pods/8dd695c3-1426-4214-a757-704c120075cb/volumes"
Oct 30 13:21:28.843649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:28.843635    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" path="/var/lib/kubelet/pods/e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48/volumes"
Oct 30 13:21:28.844017 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:28.844006    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" path="/var/lib/kubelet/pods/fc834143-970e-4e23-8140-fc53aec8ea54/volumes"
Oct 30 13:21:29.696542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:29.696512    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:30.174745 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:30.174716    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"]
Oct 30 13:21:30.367565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:30.367520    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:30.540834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:30.540693    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:31.020704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020321    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"]
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020793    5781 cpu_manager.go:414] "containerMap: removing container" podUID="72ed25d8-f936-44c6-9003-79ff1d8da414" containerName="payload-pre-processing"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020806    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="72ed25d8-f936-44c6-9003-79ff1d8da414" containerName="payload-pre-processing"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020819    5781 cpu_manager.go:414] "containerMap: removing container" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020825    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020837    5781 cpu_manager.go:414] "containerMap: removing container" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020843    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020855    5781 cpu_manager.go:414] "containerMap: removing container" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020860    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020868    5781 cpu_manager.go:414] "containerMap: removing container" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020872    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020897    5781 cpu_manager.go:414] "containerMap: removing container" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020901    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020910    5781 cpu_manager.go:414] "containerMap: removing container" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020914    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020921    5781 cpu_manager.go:414] "containerMap: removing container" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020925    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020930    5781 cpu_manager.go:414] "containerMap: removing container" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020935    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020940    5781 cpu_manager.go:414] "containerMap: removing container" podUID="75e6e1a7-832b-4a9f-8683-c66c5876c1f7" containerName="revoke-keys"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.020945    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75e6e1a7-832b-4a9f-8683-c66c5876c1f7" containerName="revoke-keys"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021008    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9f0f44c-ee5a-4d0c-91c7-d1d55fd15c48" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021019    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="035c8706-673c-4624-8b95-57c566fba8db" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021028    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="8dd695c3-1426-4214-a757-704c120075cb" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021034    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021039    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="5eb08489-1221-4ab8-9b10-df2003f17b23" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021045    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d38948d-2d60-403c-91d0-0b5f1a534e60" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021050    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="75e6e1a7-832b-4a9f-8683-c66c5876c1f7" containerName="revoke-keys"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021056    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="72ed25d8-f936-44c6-9003-79ff1d8da414" containerName="payload-pre-processing"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021061    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc834143-970e-4e23-8140-fc53aec8ea54" containerName="istio-proxy"
Oct 30 13:21:31.021094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.021068    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="65918173-d800-4142-9f9a-fe8d32d5504d" containerName="istio-proxy"
Oct 30 13:21:31.024480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.024460    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.050792 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.050767    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:21:31.050918 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.050807    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:21:31.065608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.065577    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"]
Oct 30 13:21:31.168934 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.168884    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k8hz\" (UniqueName: \"kubernetes.io/projected/9bb522b8-462e-4b77-986d-5dd2ab247e09-kube-api-access-7k8hz\") pod \"maas-api-revoke-sub-efb457e3-vrlst\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.169084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.169001    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-efb457e3-vrlst\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.169084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.169045    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-xks-ca\") pod \"maas-api-revoke-sub-efb457e3-vrlst\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.270132 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.270092    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k8hz\" (UniqueName: \"kubernetes.io/projected/9bb522b8-462e-4b77-986d-5dd2ab247e09-kube-api-access-7k8hz\") pod \"maas-api-revoke-sub-efb457e3-vrlst\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.270356 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.270155    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-efb457e3-vrlst\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.270356 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.270186    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-xks-ca\") pod \"maas-api-revoke-sub-efb457e3-vrlst\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.271004 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.270962    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-efb457e3-vrlst\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.272603 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.272580    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-xks-ca\") pod \"maas-api-revoke-sub-efb457e3-vrlst\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.296337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.296315    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k8hz\" (UniqueName: \"kubernetes.io/projected/9bb522b8-462e-4b77-986d-5dd2ab247e09-kube-api-access-7k8hz\") pod \"maas-api-revoke-sub-efb457e3-vrlst\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.340090 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.340061    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:31.643016 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.642939    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:31.643270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.643230    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"]
Oct 30 13:21:31.646686 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:21:31.646650    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bb522b8_462e_4b77_986d_5dd2ab247e09.slice/crio-d94cd73182920a9b9de529665ca28a5148fd8a6754e4441b2e6d6351c8a158e8 WatchSource:0}: Error finding container d94cd73182920a9b9de529665ca28a5148fd8a6754e4441b2e6d6351c8a158e8: Status 404 returned error can't find the container with id d94cd73182920a9b9de529665ca28a5148fd8a6754e4441b2e6d6351c8a158e8
Oct 30 13:21:31.926409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:31.926372    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst" event={"ID":"9bb522b8-462e-4b77-986d-5dd2ab247e09","Type":"ContainerStarted","Data":"d94cd73182920a9b9de529665ca28a5148fd8a6754e4441b2e6d6351c8a158e8"}
Oct 30 13:21:32.000567 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:32.000523    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:21:32.933689 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:32.933656    5781 generic.go:356] "Generic (PLEG): container finished" podID="9bb522b8-462e-4b77-986d-5dd2ab247e09" containerID="62be05d1045c41d36dc9ff81f16e6ebb0e74ea2cd5a946b1398483332e091aaf" exitCode=0
Oct 30 13:21:32.934051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:32.933748    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst" event={"ID":"9bb522b8-462e-4b77-986d-5dd2ab247e09","Type":"ContainerDied","Data":"62be05d1045c41d36dc9ff81f16e6ebb0e74ea2cd5a946b1398483332e091aaf"}
Oct 30 13:21:33.694568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:33.694532    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:34.046403 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.046378    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:34.068763 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.068743    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:34.200159 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.200112    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-openshift-ca\") pod \"9bb522b8-462e-4b77-986d-5dd2ab247e09\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") "
Oct 30 13:21:34.200358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.200176    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bb522b8-462e-4b77-986d-5dd2ab247e09-kube-api-access-7k8hz\" (UniqueName: \"kubernetes.io/projected/9bb522b8-462e-4b77-986d-5dd2ab247e09-kube-api-access-7k8hz\") pod \"9bb522b8-462e-4b77-986d-5dd2ab247e09\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") "
Oct 30 13:21:34.200358 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.200208    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-xks-ca\") pod \"9bb522b8-462e-4b77-986d-5dd2ab247e09\" (UID: \"9bb522b8-462e-4b77-986d-5dd2ab247e09\") "
Oct 30 13:21:34.200956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.200924    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-openshift-ca" pod "9bb522b8-462e-4b77-986d-5dd2ab247e09" (UID: "9bb522b8-462e-4b77-986d-5dd2ab247e09"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:34.204474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.204443    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bb522b8-462e-4b77-986d-5dd2ab247e09-kube-api-access-7k8hz" pod "9bb522b8-462e-4b77-986d-5dd2ab247e09" (UID: "9bb522b8-462e-4b77-986d-5dd2ab247e09"). InnerVolumeSpecName "kube-api-access-7k8hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:34.208411 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.208386    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-xks-ca" pod "9bb522b8-462e-4b77-986d-5dd2ab247e09" (UID: "9bb522b8-462e-4b77-986d-5dd2ab247e09"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:34.302386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.302282    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:34.302386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.302348    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k8hz\" (UniqueName: \"kubernetes.io/projected/9bb522b8-462e-4b77-986d-5dd2ab247e09-kube-api-access-7k8hz\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:34.302386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.302365    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/9bb522b8-462e-4b77-986d-5dd2ab247e09-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:34.334162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.334127    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:34.549972 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.549895    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:34.948815 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.948787    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"
Oct 30 13:21:34.949016 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.948830    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst" event={"ID":"9bb522b8-462e-4b77-986d-5dd2ab247e09","Type":"ContainerDied","Data":"d94cd73182920a9b9de529665ca28a5148fd8a6754e4441b2e6d6351c8a158e8"}
Oct 30 13:21:34.949016 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:34.948913    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d94cd73182920a9b9de529665ca28a5148fd8a6754e4441b2e6d6351c8a158e8"
Oct 30 13:21:35.175355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:35.175308    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7" podUID="7f56a9dd-efa5-4f21-b619-b6bafd9ae785" containerName="payload-processing" containerID="cri-o://759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7" gracePeriod=25
Oct 30 13:21:35.450033 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:35.449964    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:35.696264 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:35.696228    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:35.962620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:35.962589    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:36.106791 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:36.106726    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:36.275927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:36.275850    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"]
Oct 30 13:21:36.276304 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:36.276093    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy" containerID="cri-o://0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53" gracePeriod=30
Oct 30 13:21:36.285548 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:36.285524    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"]
Oct 30 13:21:36.285708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:36.285690    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy" containerID="cri-o://308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c" gracePeriod=30
Oct 30 13:21:36.340584 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:36.340556    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:37.534648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.534614    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"]
Oct 30 13:21:37.535123 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.535107    5781 cpu_manager.go:414] "containerMap: removing container" podUID="9bb522b8-462e-4b77-986d-5dd2ab247e09" containerName="revoke-keys"
Oct 30 13:21:37.535167 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.535125    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bb522b8-462e-4b77-986d-5dd2ab247e09" containerName="revoke-keys"
Oct 30 13:21:37.535228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.535220    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bb522b8-462e-4b77-986d-5dd2ab247e09" containerName="revoke-keys"
Oct 30 13:21:37.538672 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.538653    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.553223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.553207    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-ab3176-openshift-default-dockercfg-ws7g2\""
Oct 30 13:21:37.572652 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.572630    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"]
Oct 30 13:21:37.636831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.636793    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8547166-af83-43c9-aa6d-b347f45652db-istio-podinfo\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.637021 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.636853    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-istio-token\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.637021 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.636915    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8547166-af83-43c9-aa6d-b347f45652db-istiod-ca-cert\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.637136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.637034    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-certs\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.637136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.637065    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-data\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.637222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.637160    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-credential-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.637222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.637192    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-envoy\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.637380 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.637363    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.637433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.637409    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xswm4\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-kube-api-access-xswm4\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.738357 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.738327    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xswm4\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-kube-api-access-xswm4\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.738527 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.738375    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8547166-af83-43c9-aa6d-b347f45652db-istio-podinfo\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.738527 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.738503    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-istio-token\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.738615 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.738540    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8547166-af83-43c9-aa6d-b347f45652db-istiod-ca-cert\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.738862 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.738699    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-certs\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.738862 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.738737    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-data\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.738862 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.738782    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-credential-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.738862 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.738802    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-envoy\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.739119 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.738905    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.739178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.739131    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-credential-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.739237 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.739211    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-data\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.739326 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.739274    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8547166-af83-43c9-aa6d-b347f45652db-istiod-ca-cert\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.739398 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.739318    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-socket\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.739449 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.739394    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-certs\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.740982 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.740957    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8547166-af83-43c9-aa6d-b347f45652db-istio-podinfo\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.740982 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.740969    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-envoy\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.761906 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.761883    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xswm4\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-kube-api-access-xswm4\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.763530 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.763504    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-istio-token\") pod \"e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") " pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:37.851456 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:37.851400    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:38.173598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:38.173571    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"]
Oct 30 13:21:38.175550 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:21:38.175525    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8547166_af83_43c9_aa6d_b347f45652db.slice/crio-571e2405ca1835afbcb5546da870fb9cde20d4e59b81840818a4e28e796ba2fc WatchSource:0}: Error finding container 571e2405ca1835afbcb5546da870fb9cde20d4e59b81840818a4e28e796ba2fc: Status 404 returned error can't find the container with id 571e2405ca1835afbcb5546da870fb9cde20d4e59b81840818a4e28e796ba2fc
Oct 30 13:21:38.177615 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:38.177588    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:21:38.177680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:38.177648    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:21:38.177714 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:38.177681    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:21:38.974735 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:38.974695    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" event={"ID":"a8547166-af83-43c9-aa6d-b347f45652db","Type":"ContainerStarted","Data":"4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8"}
Oct 30 13:21:38.975153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:38.974752    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" event={"ID":"a8547166-af83-43c9-aa6d-b347f45652db","Type":"ContainerStarted","Data":"571e2405ca1835afbcb5546da870fb9cde20d4e59b81840818a4e28e796ba2fc"}
Oct 30 13:21:39.035311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:39.034085    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" podStartSLOduration=2.034064591 podStartE2EDuration="2.034064591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:21:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:21:39.03172627 +0000 UTC m=+1436.328943812" watchObservedRunningTime="2026-10-30 13:21:39.034064591 +0000 UTC m=+1436.331282135"
Oct 30 13:21:39.038733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:39.038701    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7" podUID="7f56a9dd-efa5-4f21-b619-b6bafd9ae785" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:39.852518 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:39.852488    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:39.853852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:39.853822    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 13:21:39.854000 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:39.853884    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" podUID="a8547166-af83-43c9-aa6d-b347f45652db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 13:21:40.852197 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:40.852162    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 13:21:40.852601 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:40.852212    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" podUID="a8547166-af83-43c9-aa6d-b347f45652db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 13:21:41.610542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.610312    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:21:41.642799 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.642773    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:21:41.650325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.650233    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:41.782232 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782202    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istiod-ca-cert\") pod \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") "
Oct 30 13:21:41.782429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782254    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-podinfo\") pod \"2c14696d-3c86-4d10-93cb-bee727caa66d\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") "
Oct 30 13:21:41.782429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782342    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-data\") pod \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") "
Oct 30 13:21:41.782429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782389    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-certs\") pod \"2c14696d-3c86-4d10-93cb-bee727caa66d\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") "
Oct 30 13:21:41.782429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782425    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-certs\") pod \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") "
Oct 30 13:21:41.782592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782459    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-data\") pod \"2c14696d-3c86-4d10-93cb-bee727caa66d\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") "
Oct 30 13:21:41.782592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782488    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c14696d-3c86-4d10-93cb-bee727caa66d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c14696d-3c86-4d10-93cb-bee727caa66d-istiod-ca-cert\") pod \"2c14696d-3c86-4d10-93cb-bee727caa66d\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") "
Oct 30 13:21:41.782592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782526    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-token\") pod \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") "
Oct 30 13:21:41.782726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782590    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-credential-socket\") pod \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") "
Oct 30 13:21:41.782726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782608    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istiod-ca-cert" pod "d1c7a654-01ce-4684-b4a4-59a62c26a26d" (UID: "d1c7a654-01ce-4684-b4a4-59a62c26a26d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:41.782726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782622    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-socket\") pod \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") "
Oct 30 13:21:41.782726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782653    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-token\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-token\") pod \"2c14696d-3c86-4d10-93cb-bee727caa66d\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") "
Oct 30 13:21:41.782908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782738    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-envoy\") pod \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") "
Oct 30 13:21:41.782908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782773    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-certs" pod "2c14696d-3c86-4d10-93cb-bee727caa66d" (UID: "2c14696d-3c86-4d10-93cb-bee727caa66d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.782908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782796    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-kube-api-access-khkmx\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-kube-api-access-khkmx\") pod \"2c14696d-3c86-4d10-93cb-bee727caa66d\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") "
Oct 30 13:21:41.782908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782789    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-data" pod "2c14696d-3c86-4d10-93cb-bee727caa66d" (UID: "2c14696d-3c86-4d10-93cb-bee727caa66d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.782908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782845    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-podinfo\") pod \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") "
Oct 30 13:21:41.782908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782860    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-data" pod "d1c7a654-01ce-4684-b4a4-59a62c26a26d" (UID: "d1c7a654-01ce-4684-b4a4-59a62c26a26d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.782908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782897    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-credential-socket\") pod \"2c14696d-3c86-4d10-93cb-bee727caa66d\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") "
Oct 30 13:21:41.782908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782876    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-certs" pod "d1c7a654-01ce-4684-b4a4-59a62c26a26d" (UID: "d1c7a654-01ce-4684-b4a4-59a62c26a26d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.783331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782928    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-socket\") pod \"2c14696d-3c86-4d10-93cb-bee727caa66d\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") "
Oct 30 13:21:41.783331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782966    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-kube-api-access-7qw9r\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-kube-api-access-7qw9r\") pod \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\" (UID: \"d1c7a654-01ce-4684-b4a4-59a62c26a26d\") "
Oct 30 13:21:41.783331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782948    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-socket" pod "d1c7a654-01ce-4684-b4a4-59a62c26a26d" (UID: "d1c7a654-01ce-4684-b4a4-59a62c26a26d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.783331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.782995    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-envoy\") pod \"2c14696d-3c86-4d10-93cb-bee727caa66d\" (UID: \"2c14696d-3c86-4d10-93cb-bee727caa66d\") "
Oct 30 13:21:41.783331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783020    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-credential-socket" pod "d1c7a654-01ce-4684-b4a4-59a62c26a26d" (UID: "d1c7a654-01ce-4684-b4a4-59a62c26a26d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.783331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783149    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-socket" pod "2c14696d-3c86-4d10-93cb-bee727caa66d" (UID: "2c14696d-3c86-4d10-93cb-bee727caa66d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.783648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783458    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783480    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783492    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783504    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783518    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783532    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783545    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.783648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783558    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.784047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.783782    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-credential-socket" pod "2c14696d-3c86-4d10-93cb-bee727caa66d" (UID: "2c14696d-3c86-4d10-93cb-bee727caa66d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.784115 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.784097    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c14696d-3c86-4d10-93cb-bee727caa66d-istiod-ca-cert" pod "2c14696d-3c86-4d10-93cb-bee727caa66d" (UID: "2c14696d-3c86-4d10-93cb-bee727caa66d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:41.785849 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.785798    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-envoy" pod "2c14696d-3c86-4d10-93cb-bee727caa66d" (UID: "2c14696d-3c86-4d10-93cb-bee727caa66d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.785849 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.785820    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-podinfo" pod "2c14696d-3c86-4d10-93cb-bee727caa66d" (UID: "2c14696d-3c86-4d10-93cb-bee727caa66d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:41.785849 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.785841    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-token" pod "d1c7a654-01ce-4684-b4a4-59a62c26a26d" (UID: "d1c7a654-01ce-4684-b4a4-59a62c26a26d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.786106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.785889    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-podinfo" pod "d1c7a654-01ce-4684-b4a4-59a62c26a26d" (UID: "d1c7a654-01ce-4684-b4a4-59a62c26a26d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:21:41.786106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.785981    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-kube-api-access-khkmx" pod "2c14696d-3c86-4d10-93cb-bee727caa66d" (UID: "2c14696d-3c86-4d10-93cb-bee727caa66d"). InnerVolumeSpecName "kube-api-access-khkmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.786106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.786065    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-kube-api-access-7qw9r" pod "d1c7a654-01ce-4684-b4a4-59a62c26a26d" (UID: "d1c7a654-01ce-4684-b4a4-59a62c26a26d"). InnerVolumeSpecName "kube-api-access-7qw9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.787149 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.787126    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-envoy" pod "d1c7a654-01ce-4684-b4a4-59a62c26a26d" (UID: "d1c7a654-01ce-4684-b4a4-59a62c26a26d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:21:41.787329 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.787307    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-token" pod "2c14696d-3c86-4d10-93cb-bee727caa66d" (UID: "2c14696d-3c86-4d10-93cb-bee727caa66d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:41.852603 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.852525    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 13:21:41.852603 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.852585    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" podUID="a8547166-af83-43c9-aa6d-b347f45652db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 13:21:41.884244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884216    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.884244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884240    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c14696d-3c86-4d10-93cb-bee727caa66d-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.884244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884252    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.884491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884263    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.884491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884271    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.884491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884279    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khkmx\" (UniqueName: \"kubernetes.io/projected/2c14696d-3c86-4d10-93cb-bee727caa66d-kube-api-access-khkmx\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.884491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884297    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c7a654-01ce-4684-b4a4-59a62c26a26d-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.884491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884306    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.884491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884314    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qw9r\" (UniqueName: \"kubernetes.io/projected/d1c7a654-01ce-4684-b4a4-59a62c26a26d-kube-api-access-7qw9r\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.884491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.884322    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c14696d-3c86-4d10-93cb-bee727caa66d-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:41.994560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.994525    5781 generic.go:356] "Generic (PLEG): container finished" podID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerID="0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53" exitCode=0
Oct 30 13:21:41.994731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.994573    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"
Oct 30 13:21:41.994731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.994601    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" event={"ID":"2c14696d-3c86-4d10-93cb-bee727caa66d","Type":"ContainerDied","Data":"0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53"}
Oct 30 13:21:41.994731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.994665    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j" event={"ID":"2c14696d-3c86-4d10-93cb-bee727caa66d","Type":"ContainerDied","Data":"ad6010048228e858c085217f536eac9169faf4529429f431700c36de91730dd6"}
Oct 30 13:21:41.994731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.994696    5781 scope.go:122] "RemoveContainer" containerID="0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53"
Oct 30 13:21:41.996331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.996304    5781 generic.go:356] "Generic (PLEG): container finished" podID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerID="308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c" exitCode=0
Oct 30 13:21:41.996487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.996373    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"
Oct 30 13:21:41.996487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.996401    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" event={"ID":"d1c7a654-01ce-4684-b4a4-59a62c26a26d","Type":"ContainerDied","Data":"308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c"}
Oct 30 13:21:41.996487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:41.996448    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt" event={"ID":"d1c7a654-01ce-4684-b4a4-59a62c26a26d","Type":"ContainerDied","Data":"451cb23e5b6cdb5f684dce1a1dbcb6169bc5e29609770b22775fb5ec10e64235"}
Oct 30 13:21:42.006443 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.006425    5781 scope.go:122] "RemoveContainer" containerID="0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53"
Oct 30 13:21:42.006701 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:42.006684    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53\": container with ID starting with 0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53 not found: ID does not exist" containerID="0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53"
Oct 30 13:21:42.006776 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.006714    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53"} err="failed to get container status \"0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53\": rpc error: code = NotFound desc = could not find container \"0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53\": container with ID starting with 0881bc742e4f7ed654905a39fb2036b8b0a45c7d1784745bb06260ac0b52ac53 not found: ID does not exist"
Oct 30 13:21:42.006776 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.006737    5781 scope.go:122] "RemoveContainer" containerID="308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c"
Oct 30 13:21:42.040690 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.040669    5781 scope.go:122] "RemoveContainer" containerID="308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c"
Oct 30 13:21:42.041024 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:42.041005    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c\": container with ID starting with 308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c not found: ID does not exist" containerID="308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c"
Oct 30 13:21:42.041076 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.041034    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c"} err="failed to get container status \"308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c\": rpc error: code = NotFound desc = could not find container \"308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c\": container with ID starting with 308c03c652f139e182d6776898637aa840b838a8401e9770dd5752764183ae5c not found: ID does not exist"
Oct 30 13:21:42.065316 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.065270    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"]
Oct 30 13:21:42.070550 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.070520    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-gbq7j"]
Oct 30 13:21:42.107381 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.107312    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"]
Oct 30 13:21:42.126394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.112580    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-88f6a4-openshift-default-64b46ffd7-shspt"]
Oct 30 13:21:42.843720 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.843691    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" path="/var/lib/kubelet/pods/2c14696d-3c86-4d10-93cb-bee727caa66d/volumes"
Oct 30 13:21:42.844143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.844130    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" path="/var/lib/kubelet/pods/d1c7a654-01ce-4684-b4a4-59a62c26a26d/volumes"
Oct 30 13:21:42.852732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.852706    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 13:21:42.853138 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:42.852755    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" podUID="a8547166-af83-43c9-aa6d-b347f45652db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 13:21:43.851952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:43.851917    5781 patch_prober.go:29] interesting pod/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused" start-of-body=
Oct 30 13:21:43.852140 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:43.851972    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" podUID="a8547166-af83-43c9-aa6d-b347f45652db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.115:15021/healthz/ready\": dial tcp 10.134.0.115:15021: connect: connection refused"
Oct 30 13:21:44.654853 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:44.654820    5781 scope.go:122] "RemoveContainer" containerID="d838f6e9664f6c0b59a8e8ecac64fa0a6096f2fee62783096aa439bc91c6526c"
Oct 30 13:21:44.664045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:44.664020    5781 scope.go:122] "RemoveContainer" containerID="e5941838859a6fc7cdf7283c5fe5d3e37ac740be7a4daae86ae71efe9c161476"
Oct 30 13:21:44.672308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:44.672267    5781 scope.go:122] "RemoveContainer" containerID="ddffc9dab4ca30c3bf88c46cb5ed78eafa9aa24e0f9a311e936735cbabf42cb1"
Oct 30 13:21:44.682491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:44.682474    5781 scope.go:122] "RemoveContainer" containerID="83902f869ffa19546c1426c665e05004721844848f5c73eae1be76903efe5f5d"
Oct 30 13:21:45.273032 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:45.273005    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:45.273345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:45.273318    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:45.274138 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:45.274110    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:21:46.194768 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:46.194735    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:46.624016 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:46.623936    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:46.778624 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:46.778587    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:47.292983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:47.292945    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:49.023866 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:49.023823    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7" podUID="7f56a9dd-efa5-4f21-b619-b6bafd9ae785" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:21:51.441135 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:51.441103    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:54.364876 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:54.364839    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:54.527045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:54.527013    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:54.686036 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:54.685999    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"]
Oct 30 13:21:54.686328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:54.686260    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" podUID="a8547166-af83-43c9-aa6d-b347f45652db" containerName="istio-proxy" containerID="cri-o://4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8" gracePeriod=30
Oct 30 13:21:54.758837 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:54.758804    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:21:55.422794 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:55.422768    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:21:55.512864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:55.512796    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-plugins-config-volume\") pod \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\" (UID: \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\") "
Oct 30 13:21:55.512977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:55.512917    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-kube-api-access-jgjg5\" (UniqueName: \"kubernetes.io/projected/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-kube-api-access-jgjg5\") pod \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\" (UID: \"7f56a9dd-efa5-4f21-b619-b6bafd9ae785\") "
Oct 30 13:21:55.513166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:55.513146    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-plugins-config-volume" pod "7f56a9dd-efa5-4f21-b619-b6bafd9ae785" (UID: "7f56a9dd-efa5-4f21-b619-b6bafd9ae785"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:21:55.515006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:55.514972    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-kube-api-access-jgjg5" pod "7f56a9dd-efa5-4f21-b619-b6bafd9ae785" (UID: "7f56a9dd-efa5-4f21-b619-b6bafd9ae785"). InnerVolumeSpecName "kube-api-access-jgjg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:55.614219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:55.614186    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:55.614219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:55.614212    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgjg5\" (UniqueName: \"kubernetes.io/projected/7f56a9dd-efa5-4f21-b619-b6bafd9ae785-kube-api-access-jgjg5\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:56.094042 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.094012    5781 generic.go:356] "Generic (PLEG): container finished" podID="7f56a9dd-efa5-4f21-b619-b6bafd9ae785" containerID="759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7" exitCode=0
Oct 30 13:21:56.094231 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.094060    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"
Oct 30 13:21:56.094231 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.094091    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7" event={"ID":"7f56a9dd-efa5-4f21-b619-b6bafd9ae785","Type":"ContainerDied","Data":"759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7"}
Oct 30 13:21:56.094231 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.094146    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7" event={"ID":"7f56a9dd-efa5-4f21-b619-b6bafd9ae785","Type":"ContainerDied","Data":"8c6a792558f0d9b190f7e3b063746cc0fafe1178462110e5802779cc09957659"}
Oct 30 13:21:56.094231 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.094167    5781 scope.go:122] "RemoveContainer" containerID="759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7"
Oct 30 13:21:56.103499 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.103481    5781 scope.go:122] "RemoveContainer" containerID="759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7"
Oct 30 13:21:56.103769 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:56.103750    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7\": container with ID starting with 759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7 not found: ID does not exist" containerID="759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7"
Oct 30 13:21:56.103826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.103782    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7"} err="failed to get container status \"759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7\": rpc error: code = NotFound desc = could not find container \"759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7\": container with ID starting with 759fd2ef8495ba726f81a97132c74caf023a0ef498616634e6a54659eaff39d7 not found: ID does not exist"
Oct 30 13:21:56.150610 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.150587    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"]
Oct 30 13:21:56.161946 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.161926    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-ab3176-57bbf887f4-7vpt7"]
Oct 30 13:21:56.840231 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:56.840202    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f56a9dd-efa5-4f21-b619-b6bafd9ae785" path="/var/lib/kubelet/pods/7f56a9dd-efa5-4f21-b619-b6bafd9ae785/volumes"
Oct 30 13:21:57.350541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.350500    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-nkm7w"]
Oct 30 13:21:57.350785 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.350728    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w" podUID="81b244f2-6fc5-45b9-ba63-08427c71baed" containerName="manager" containerID="cri-o://4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c" gracePeriod=30
Oct 30 13:21:57.600981 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.600927    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:21:57.733088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.733052    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-webhook-cert\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-webhook-cert\") pod \"81b244f2-6fc5-45b9-ba63-08427c71baed\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") "
Oct 30 13:21:57.733275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.733142    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-metrics-certs\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-metrics-certs\") pod \"81b244f2-6fc5-45b9-ba63-08427c71baed\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") "
Oct 30 13:21:57.733275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.733253    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81b244f2-6fc5-45b9-ba63-08427c71baed-kube-api-access-hdmbd\" (UniqueName: \"kubernetes.io/projected/81b244f2-6fc5-45b9-ba63-08427c71baed-kube-api-access-hdmbd\") pod \"81b244f2-6fc5-45b9-ba63-08427c71baed\" (UID: \"81b244f2-6fc5-45b9-ba63-08427c71baed\") "
Oct 30 13:21:57.735538 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.735503    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b244f2-6fc5-45b9-ba63-08427c71baed-kube-api-access-hdmbd" pod "81b244f2-6fc5-45b9-ba63-08427c71baed" (UID: "81b244f2-6fc5-45b9-ba63-08427c71baed"). InnerVolumeSpecName "kube-api-access-hdmbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:21:57.735668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.735598    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-metrics-certs" pod "81b244f2-6fc5-45b9-ba63-08427c71baed" (UID: "81b244f2-6fc5-45b9-ba63-08427c71baed"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:57.735668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.735625    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-webhook-cert" pod "81b244f2-6fc5-45b9-ba63-08427c71baed" (UID: "81b244f2-6fc5-45b9-ba63-08427c71baed"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:21:57.834013 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.833990    5781 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-metrics-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:57.834013 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.834012    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdmbd\" (UniqueName: \"kubernetes.io/projected/81b244f2-6fc5-45b9-ba63-08427c71baed-kube-api-access-hdmbd\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:57.834145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:57.834022    5781 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81b244f2-6fc5-45b9-ba63-08427c71baed-webhook-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:21:58.109527 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.109494    5781 generic.go:356] "Generic (PLEG): container finished" podID="81b244f2-6fc5-45b9-ba63-08427c71baed" containerID="4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c" exitCode=0
Oct 30 13:21:58.109892 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.109537    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w"
Oct 30 13:21:58.109892 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.109576    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w" event={"ID":"81b244f2-6fc5-45b9-ba63-08427c71baed","Type":"ContainerDied","Data":"4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c"}
Oct 30 13:21:58.109892 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.109622    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-nkm7w" event={"ID":"81b244f2-6fc5-45b9-ba63-08427c71baed","Type":"ContainerDied","Data":"3e368bc78a9492d84f62e2ddbafa12179923b997c08aefb5a639b843c02d4ec5"}
Oct 30 13:21:58.109892 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.109645    5781 scope.go:122] "RemoveContainer" containerID="4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c"
Oct 30 13:21:58.127674 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.127654    5781 scope.go:122] "RemoveContainer" containerID="4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c"
Oct 30 13:21:58.127930 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:21:58.127913    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c\": container with ID starting with 4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c not found: ID does not exist" containerID="4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c"
Oct 30 13:21:58.127966 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.127939    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c"} err="failed to get container status \"4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c\": rpc error: code = NotFound desc = could not find container \"4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c\": container with ID starting with 4149c4e408cc9581178a54e5c57fc7a59ce91e756c6ec7903435daa73514a28c not found: ID does not exist"
Oct 30 13:21:58.161105 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.161083    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-nkm7w"]
Oct 30 13:21:58.172596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.172576    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-nkm7w"]
Oct 30 13:21:58.840195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.840149    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81b244f2-6fc5-45b9-ba63-08427c71baed" path="/var/lib/kubelet/pods/81b244f2-6fc5-45b9-ba63-08427c71baed/volumes"
Oct 30 13:21:58.842265 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842243    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-whr85"]
Oct 30 13:21:58.842675 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842660    5781 cpu_manager.go:414] "containerMap: removing container" podUID="7f56a9dd-efa5-4f21-b619-b6bafd9ae785" containerName="payload-processing"
Oct 30 13:21:58.842722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842675    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f56a9dd-efa5-4f21-b619-b6bafd9ae785" containerName="payload-processing"
Oct 30 13:21:58.842722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842700    5781 cpu_manager.go:414] "containerMap: removing container" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy"
Oct 30 13:21:58.842722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842704    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy"
Oct 30 13:21:58.842722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842714    5781 cpu_manager.go:414] "containerMap: removing container" podUID="81b244f2-6fc5-45b9-ba63-08427c71baed" containerName="manager"
Oct 30 13:21:58.842722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842718    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81b244f2-6fc5-45b9-ba63-08427c71baed" containerName="manager"
Oct 30 13:21:58.842896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842731    5781 cpu_manager.go:414] "containerMap: removing container" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy"
Oct 30 13:21:58.842896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842735    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy"
Oct 30 13:21:58.842896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842810    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c14696d-3c86-4d10-93cb-bee727caa66d" containerName="istio-proxy"
Oct 30 13:21:58.842896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842820    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="81b244f2-6fc5-45b9-ba63-08427c71baed" containerName="manager"
Oct 30 13:21:58.842896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842826    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1c7a654-01ce-4684-b4a4-59a62c26a26d" containerName="istio-proxy"
Oct 30 13:21:58.842896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.842835    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f56a9dd-efa5-4f21-b619-b6bafd9ae785" containerName="payload-processing"
Oct 30 13:21:58.847559 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.847541    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:58.872145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.872130    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 13:21:58.883720 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.883703    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 13:21:58.883949 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.883931    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kvhzq\""
Oct 30 13:21:58.883997 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.883952    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 13:21:58.888713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.888693    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-whr85"]
Oct 30 13:21:58.943469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.943448    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-metrics-certs\") pod \"maas-controller-5444d7fdb7-whr85\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") " pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:58.943980 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.943958    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdckp\" (UniqueName: \"kubernetes.io/projected/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-kube-api-access-vdckp\") pod \"maas-controller-5444d7fdb7-whr85\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") " pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:58.944058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:58.943990    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-webhook-cert\") pod \"maas-controller-5444d7fdb7-whr85\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") " pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:59.045281 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:59.045246    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdckp\" (UniqueName: \"kubernetes.io/projected/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-kube-api-access-vdckp\") pod \"maas-controller-5444d7fdb7-whr85\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") " pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:59.045424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:59.045307    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-webhook-cert\") pod \"maas-controller-5444d7fdb7-whr85\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") " pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:59.045424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:59.045388    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-metrics-certs\") pod \"maas-controller-5444d7fdb7-whr85\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") " pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:59.047709 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:59.047687    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-webhook-cert\") pod \"maas-controller-5444d7fdb7-whr85\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") " pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:59.047802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:59.047744    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-metrics-certs\") pod \"maas-controller-5444d7fdb7-whr85\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") " pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:59.067638 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:59.067616    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdckp\" (UniqueName: \"kubernetes.io/projected/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-kube-api-access-vdckp\") pod \"maas-controller-5444d7fdb7-whr85\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") " pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:59.161089 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:59.161029    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:21:59.463108 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:59.463074    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-whr85"]
Oct 30 13:21:59.466441 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:21:59.466419    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad11a1a_809c_482b_ba1d_5aeaac0784c9.slice/crio-9d939873dc538fa6c32e6a1378e464d1792602143a39789feba01618cde2a087 WatchSource:0}: Error finding container 9d939873dc538fa6c32e6a1378e464d1792602143a39789feba01618cde2a087: Status 404 returned error can't find the container with id 9d939873dc538fa6c32e6a1378e464d1792602143a39789feba01618cde2a087
Oct 30 13:21:59.931234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:21:59.931213    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:22:00.055149 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055084    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-istio-token\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-istio-token\") pod \"a8547166-af83-43c9-aa6d-b347f45652db\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") "
Oct 30 13:22:00.055149 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055135    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-data\") pod \"a8547166-af83-43c9-aa6d-b347f45652db\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") "
Oct 30 13:22:00.055365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055167    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a8547166-af83-43c9-aa6d-b347f45652db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8547166-af83-43c9-aa6d-b347f45652db-istio-podinfo\") pod \"a8547166-af83-43c9-aa6d-b347f45652db\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") "
Oct 30 13:22:00.055365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055199    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-credential-socket\") pod \"a8547166-af83-43c9-aa6d-b347f45652db\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") "
Oct 30 13:22:00.055365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055238    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-kube-api-access-xswm4\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-kube-api-access-xswm4\") pod \"a8547166-af83-43c9-aa6d-b347f45652db\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") "
Oct 30 13:22:00.055365 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055311    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-envoy\") pod \"a8547166-af83-43c9-aa6d-b347f45652db\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") "
Oct 30 13:22:00.055573 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055521    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-credential-socket" pod "a8547166-af83-43c9-aa6d-b347f45652db" (UID: "a8547166-af83-43c9-aa6d-b347f45652db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.055573 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055532    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8547166-af83-43c9-aa6d-b347f45652db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8547166-af83-43c9-aa6d-b347f45652db-istiod-ca-cert\") pod \"a8547166-af83-43c9-aa6d-b347f45652db\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") "
Oct 30 13:22:00.055573 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055557    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-data" pod "a8547166-af83-43c9-aa6d-b347f45652db" (UID: "a8547166-af83-43c9-aa6d-b347f45652db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.055709 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055603    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-certs\") pod \"a8547166-af83-43c9-aa6d-b347f45652db\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") "
Oct 30 13:22:00.055709 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055648    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-socket\") pod \"a8547166-af83-43c9-aa6d-b347f45652db\" (UID: \"a8547166-af83-43c9-aa6d-b347f45652db\") "
Oct 30 13:22:00.055979 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.055916    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-certs" pod "a8547166-af83-43c9-aa6d-b347f45652db" (UID: "a8547166-af83-43c9-aa6d-b347f45652db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.056052 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.056010    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8547166-af83-43c9-aa6d-b347f45652db-istiod-ca-cert" pod "a8547166-af83-43c9-aa6d-b347f45652db" (UID: "a8547166-af83-43c9-aa6d-b347f45652db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:22:00.056125 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.056095    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.056125 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.056116    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.056238 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.056129    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.056342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.056257    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-socket" pod "a8547166-af83-43c9-aa6d-b347f45652db" (UID: "a8547166-af83-43c9-aa6d-b347f45652db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.057621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.057587    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-istio-token" pod "a8547166-af83-43c9-aa6d-b347f45652db" (UID: "a8547166-af83-43c9-aa6d-b347f45652db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:22:00.057930 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.057795    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-envoy" pod "a8547166-af83-43c9-aa6d-b347f45652db" (UID: "a8547166-af83-43c9-aa6d-b347f45652db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:22:00.058183 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.058157    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-kube-api-access-xswm4" pod "a8547166-af83-43c9-aa6d-b347f45652db" (UID: "a8547166-af83-43c9-aa6d-b347f45652db"). InnerVolumeSpecName "kube-api-access-xswm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:22:00.058318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.058260    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8547166-af83-43c9-aa6d-b347f45652db-istio-podinfo" pod "a8547166-af83-43c9-aa6d-b347f45652db" (UID: "a8547166-af83-43c9-aa6d-b347f45652db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:22:00.124001 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.123969    5781 generic.go:356] "Generic (PLEG): container finished" podID="a8547166-af83-43c9-aa6d-b347f45652db" containerID="4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8" exitCode=0
Oct 30 13:22:00.124195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.124019    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"
Oct 30 13:22:00.124195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.124040    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" event={"ID":"a8547166-af83-43c9-aa6d-b347f45652db","Type":"ContainerDied","Data":"4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8"}
Oct 30 13:22:00.124195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.124093    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm" event={"ID":"a8547166-af83-43c9-aa6d-b347f45652db","Type":"ContainerDied","Data":"571e2405ca1835afbcb5546da870fb9cde20d4e59b81840818a4e28e796ba2fc"}
Oct 30 13:22:00.124195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.124126    5781 scope.go:122] "RemoveContainer" containerID="4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8"
Oct 30 13:22:00.126493 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.126462    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-whr85" event={"ID":"2ad11a1a-809c-482b-ba1d-5aeaac0784c9","Type":"ContainerStarted","Data":"a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04"}
Oct 30 13:22:00.126618 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.126498    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-whr85" event={"ID":"2ad11a1a-809c-482b-ba1d-5aeaac0784c9","Type":"ContainerStarted","Data":"9d939873dc538fa6c32e6a1378e464d1792602143a39789feba01618cde2a087"}
Oct 30 13:22:00.126618 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.126535    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:22:00.134583 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.134570    5781 scope.go:122] "RemoveContainer" containerID="4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8"
Oct 30 13:22:00.134846 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:22:00.134829    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8\": container with ID starting with 4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8 not found: ID does not exist" containerID="4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8"
Oct 30 13:22:00.134886 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.134854    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8"} err="failed to get container status \"4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8\": rpc error: code = NotFound desc = could not find container \"4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8\": container with ID starting with 4d5680762d51e0b9ede018e874780336c53b40b91d08001268dc3dc02c3affe8 not found: ID does not exist"
Oct 30 13:22:00.157344 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.157322    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xswm4\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-kube-api-access-xswm4\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.157344 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.157342    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.157468 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.157351    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8547166-af83-43c9-aa6d-b347f45652db-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.157468 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.157359    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8547166-af83-43c9-aa6d-b347f45652db-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.157468 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.157367    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8547166-af83-43c9-aa6d-b347f45652db-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.157468 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.157376    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8547166-af83-43c9-aa6d-b347f45652db-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:00.175983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.175953    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"]
Oct 30 13:22:00.178349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.178326    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ab3176-openshift-default-859bd4cb9c-nl2tm"]
Oct 30 13:22:00.217847 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.217801    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5444d7fdb7-whr85" podStartSLOduration=1.817627155 podStartE2EDuration="2.217785582s" totalImagesPullingTime="400.158427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:21:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:22:00.215784169 +0000 UTC m=+1457.513001711" watchObservedRunningTime="2026-10-30 13:22:00.217785582 +0000 UTC m=+1457.515003128"
Oct 30 13:22:00.840379 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:00.840346    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8547166-af83-43c9-aa6d-b347f45652db" path="/var/lib/kubelet/pods/a8547166-af83-43c9-aa6d-b347f45652db/volumes"
Oct 30 13:22:01.197243 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:01.197215    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:22:01.465576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:01.465489    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:22:01.994653 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:22:01.994615    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:22:02.867012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:02.866977    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:22:09.354742 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:09.354707    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:22:11.136390 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:11.136354    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:22:15.032193 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:15.032161    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"]
Oct 30 13:22:15.043449 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:15.043425    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b226dff0-rb6p7"]
Oct 30 13:22:16.840064 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:16.840032    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d5ceb6f-ff43-4a28-a15a-560beef6112c" path="/var/lib/kubelet/pods/1d5ceb6f-ff43-4a28-a15a-560beef6112c/volumes"
Oct 30 13:22:21.031864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.031832    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"]
Oct 30 13:22:21.032300 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.032276    5781 cpu_manager.go:414] "containerMap: removing container" podUID="a8547166-af83-43c9-aa6d-b347f45652db" containerName="istio-proxy"
Oct 30 13:22:21.032349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.032304    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8547166-af83-43c9-aa6d-b347f45652db" containerName="istio-proxy"
Oct 30 13:22:21.032399 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.032390    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8547166-af83-43c9-aa6d-b347f45652db" containerName="istio-proxy"
Oct 30 13:22:21.036927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.036905    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.051324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.051302    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"]
Oct 30 13:22:21.054873 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.054857    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.060195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.060169    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5td7d\""
Oct 30 13:22:21.069846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.069820    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"]
Oct 30 13:22:21.079442 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.079418    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"]
Oct 30 13:22:21.083919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.083899    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.100312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.100278    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"]
Oct 30 13:22:21.108790 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.108769    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"]
Oct 30 13:22:21.149662 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.149633    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.149662 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.149662    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.149818 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.149690    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.149818 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.149741    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/538b556d-43d4-43e1-9040-224227990f0d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.149896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.149864    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.149933 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.149923    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.149979 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.149965    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/538b556d-43d4-43e1-9040-224227990f0d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.150027 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150013    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.150066 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150042    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpx8\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-kube-api-access-kmpx8\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.150066 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150061    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.150144 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150128    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.150196 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150182    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.150234 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150218    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.150274 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150243    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwjvj\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-kube-api-access-nwjvj\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.150345 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150302    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a8de30d-9574-4bdf-ba11-c99dba512e69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.150383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150364    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.150434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150382    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.150434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.150413    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.251600 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251571    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.251600 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251603    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwjvj\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-kube-api-access-nwjvj\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.251797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251620    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a8de30d-9574-4bdf-ba11-c99dba512e69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.251797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251643    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/273ba69d-816e-4281-84f6-77f9c2180692-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.251797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251668    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.251797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251683    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.251797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251744    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.251797 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251773    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.252058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251800    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/273ba69d-816e-4281-84f6-77f9c2180692-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.252058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251821    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.252058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251837    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.252058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251907    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4rc4\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-kube-api-access-w4rc4\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.252058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251944    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.252058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.251995    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/538b556d-43d4-43e1-9040-224227990f0d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.252389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252136    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.252389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252174    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.252389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252218    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.252389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252221    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.252389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252316    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.252389 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252342    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.252703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252478    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/538b556d-43d4-43e1-9040-224227990f0d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.252703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252521    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.252703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252525    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.252703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252573    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpx8\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-kube-api-access-kmpx8\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.252703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252595    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.252703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252630    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.252703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252665    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.252703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252691    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a8de30d-9574-4bdf-ba11-c99dba512e69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.253112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252693    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.253112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252766    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.253218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252848    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.253277 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252859    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.253277 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.252939    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.253277 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.253119    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.253426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.253127    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.253426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.253230    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/538b556d-43d4-43e1-9040-224227990f0d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.253426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.253264    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.254694 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.254665    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.254800 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.254665    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.254800 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.254721    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/538b556d-43d4-43e1-9040-224227990f0d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.254990 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.254974    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.274179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.274156    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.274490 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.274471    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.274548 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.274470    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwjvj\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-kube-api-access-nwjvj\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-pmfxx\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.274762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.274745    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpx8\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-kube-api-access-kmpx8\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-zpz7d\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.350971 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.350916    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:21.354280 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354261    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/273ba69d-816e-4281-84f6-77f9c2180692-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354329    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354352    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/273ba69d-816e-4281-84f6-77f9c2180692-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354379    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4rc4\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-kube-api-access-w4rc4\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354547 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354405    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354547 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354428    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354547 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354480    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354547 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354533    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354555    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354659    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-credential-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.354832 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.354805    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-certs\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.355540 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.355095    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-data\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.355540 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.355123    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-socket\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.355540 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.355261    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/273ba69d-816e-4281-84f6-77f9c2180692-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.356726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.356706    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/273ba69d-816e-4281-84f6-77f9c2180692-istio-podinfo\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.356945 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.356930    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-envoy\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.367109 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.367090    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:21.380668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.380648    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-istio-token\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.381486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.381468    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4rc4\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-kube-api-access-w4rc4\") pod \"maas-default-gateway-openshift-default-c9ffffc7f-jhs8w\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") " pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.403507 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.403478    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:21.686838 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.686812    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"]
Oct 30 13:22:21.693444 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:22:21.693416    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538b556d_43d4_43e1_9040_224227990f0d.slice/crio-8434932f14c7f4ba1b6d95c43a7c698417ca4dd16f8b363ba9b8c37d83249d5e WatchSource:0}: Error finding container 8434932f14c7f4ba1b6d95c43a7c698417ca4dd16f8b363ba9b8c37d83249d5e: Status 404 returned error can't find the container with id 8434932f14c7f4ba1b6d95c43a7c698417ca4dd16f8b363ba9b8c37d83249d5e
Oct 30 13:22:21.695497 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.695463    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:22:21.695595 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.695524    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:22:21.695595 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.695550    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:22:21.712223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.712140    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"]
Oct 30 13:22:21.714835 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:22:21.714806    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8de30d_9574_4bdf_ba11_c99dba512e69.slice/crio-64a806c9ebad9aca1bb25fbfe78035495e44c203f6ec7611e3190e64ee3e10f3 WatchSource:0}: Error finding container 64a806c9ebad9aca1bb25fbfe78035495e44c203f6ec7611e3190e64ee3e10f3: Status 404 returned error can't find the container with id 64a806c9ebad9aca1bb25fbfe78035495e44c203f6ec7611e3190e64ee3e10f3
Oct 30 13:22:21.722866 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.722834    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:22:21.722956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.722905    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:22:21.722956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.722946    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:22:21.759427 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.759403    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"]
Oct 30 13:22:21.764812 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:22:21.764785    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod273ba69d_816e_4281_84f6_77f9c2180692.slice/crio-2df742ddddd4eb6565d8470347edf1ee3612cbeac442f7215a735699b5c0944f WatchSource:0}: Error finding container 2df742ddddd4eb6565d8470347edf1ee3612cbeac442f7215a735699b5c0944f: Status 404 returned error can't find the container with id 2df742ddddd4eb6565d8470347edf1ee3612cbeac442f7215a735699b5c0944f
Oct 30 13:22:21.768045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.768013    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:22:21.768145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.768074    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:22:21.768145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:21.768100    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:22:22.274921 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.274885    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" event={"ID":"8a8de30d-9574-4bdf-ba11-c99dba512e69","Type":"ContainerStarted","Data":"daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459"}
Oct 30 13:22:22.275497 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.275471    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" event={"ID":"8a8de30d-9574-4bdf-ba11-c99dba512e69","Type":"ContainerStarted","Data":"64a806c9ebad9aca1bb25fbfe78035495e44c203f6ec7611e3190e64ee3e10f3"}
Oct 30 13:22:22.278996 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.278967    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" event={"ID":"273ba69d-816e-4281-84f6-77f9c2180692","Type":"ContainerStarted","Data":"a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a"}
Oct 30 13:22:22.279202 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.279185    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" event={"ID":"273ba69d-816e-4281-84f6-77f9c2180692","Type":"ContainerStarted","Data":"2df742ddddd4eb6565d8470347edf1ee3612cbeac442f7215a735699b5c0944f"}
Oct 30 13:22:22.291677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.290840    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" event={"ID":"538b556d-43d4-43e1-9040-224227990f0d","Type":"ContainerStarted","Data":"fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6"}
Oct 30 13:22:22.291677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.290904    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" event={"ID":"538b556d-43d4-43e1-9040-224227990f0d","Type":"ContainerStarted","Data":"8434932f14c7f4ba1b6d95c43a7c698417ca4dd16f8b363ba9b8c37d83249d5e"}
Oct 30 13:22:22.365032 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.365002    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:22.372702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.369889    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 13:22:22.372702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.369947    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 13:22:22.372702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.371839    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:22.375584 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.375485    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:22:22.375584 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.375530    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 13:22:22.395340 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.382962    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podStartSLOduration=1.382945595 podStartE2EDuration="1.382945595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:22:22.334801674 +0000 UTC m=+1479.632019216" watchObservedRunningTime="2026-10-30 13:22:22.382945595 +0000 UTC m=+1479.680163138"
Oct 30 13:22:22.398450 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.396486    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podStartSLOduration=1.396466575 podStartE2EDuration="1.396466575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:22:22.39575029 +0000 UTC m=+1479.692967833" watchObservedRunningTime="2026-10-30 13:22:22.396466575 +0000 UTC m=+1479.693684120"
Oct 30 13:22:22.408233 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.407650    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:22.409452 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.409421    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:22:22.409568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.409475    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 13:22:22.456311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:22.455313    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podStartSLOduration=1.455277894 podStartE2EDuration="1.455277894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:22:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:22:22.452761505 +0000 UTC m=+1479.749979050" watchObservedRunningTime="2026-10-30 13:22:22.455277894 +0000 UTC m=+1479.752495437"
Oct 30 13:22:23.373325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:23.373173    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:22:23.373325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:23.373230    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 13:22:23.373877 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:23.373787    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 13:22:23.373877 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:23.373822    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 13:22:23.404687 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:23.404654    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:22:23.404837 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:23.404711    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 13:22:24.355312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:24.353542    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 13:22:24.355312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:24.353599    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 13:22:24.374308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:24.374261    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:22:24.374887 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:24.374860    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 13:22:24.404494 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:24.404468    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:22:24.404608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:24.404511    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 13:22:25.353311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:25.352886    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 13:22:25.353311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:25.352944    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 13:22:25.373310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:25.373216    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:22:25.373310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:25.373280    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 13:22:25.404182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:25.404151    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:22:25.404565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:25.404199    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 13:22:26.359070 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:26.359030    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 13:22:26.359382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:26.359351    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 13:22:26.382308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:26.382252    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:22:26.382579 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:26.382551    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 13:22:26.404151 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:26.404103    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:22:26.404306 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:26.404164    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 13:22:27.403444 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:27.403408    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:22:27.403739 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:27.403687    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 13:22:27.403914 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:27.403780    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 13:22:27.404091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:27.404032    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 13:22:27.404185 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:27.404091    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:22:27.404461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:27.404134    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 13:22:28.090710 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:28.090674    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:22:28.351369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:28.351266    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 13:22:28.351369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:28.351353    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 13:22:28.368052 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:28.368027    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused" start-of-body=
Oct 30 13:22:28.368168 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:28.368076    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": dial tcp 10.134.0.118:15021: connect: connection refused"
Oct 30 13:22:28.404669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:28.404643    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 13:22:28.405051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:28.404691    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 13:22:29.056645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:29.056611    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"]
Oct 30 13:22:29.067877 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:29.067852    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9ba3b553-mgx57"]
Oct 30 13:22:30.361870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:30.361835    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:30.362411 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:30.361900    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:30.369247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:30.369217    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:30.369392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:30.369281    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:30.405212 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:30.405164    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:30.405434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:30.405258    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:30.841083 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:30.841042    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57ed8a6b-bfa7-4648-87cc-20e225624628" path="/var/lib/kubelet/pods/57ed8a6b-bfa7-4648-87cc-20e225624628/volumes"
Oct 30 13:22:31.405438 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:31.405408    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:31.405855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:31.405491    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:31.420388 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:31.420363    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:31.420589 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:31.420566    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:31.449023 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:31.447357    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:31.449023 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:31.447405    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:31.794258 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:31.794223    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:22:32.005319 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:22:32.005258    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:22:32.447406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.447372    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:32.447960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.447931    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:32.469414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.469361    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:32.469630 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.469608    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.118:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:32.489419 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.489391    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:22:32.489590 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.489569    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:22:32.863856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.859237    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:32.865153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.865117    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:32.900355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.900278    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:32.900847 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.900826    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:32.901541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.901514    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:22:32.907236 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.907208    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:22:32.988179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.988135    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:32.989328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.989218    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:32.990119 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:32.990097    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:22:33.454599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.454564    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:22:33.573051 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.573017    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-whr85"]
Oct 30 13:22:33.573276 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.573249    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5444d7fdb7-whr85" podUID="2ad11a1a-809c-482b-ba1d-5aeaac0784c9" containerName="manager" containerID="cri-o://a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04" gracePeriod=30
Oct 30 13:22:33.824351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.824328    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:22:33.917357 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.917314    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-kube-api-access-vdckp\" (UniqueName: \"kubernetes.io/projected/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-kube-api-access-vdckp\") pod \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") "
Oct 30 13:22:33.917536 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.917363    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-metrics-certs\") pod \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") "
Oct 30 13:22:33.917536 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.917417    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-webhook-cert\") pod \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\" (UID: \"2ad11a1a-809c-482b-ba1d-5aeaac0784c9\") "
Oct 30 13:22:33.919403 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.919373    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-webhook-cert" pod "2ad11a1a-809c-482b-ba1d-5aeaac0784c9" (UID: "2ad11a1a-809c-482b-ba1d-5aeaac0784c9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:22:33.919517 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.919399    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-metrics-certs" pod "2ad11a1a-809c-482b-ba1d-5aeaac0784c9" (UID: "2ad11a1a-809c-482b-ba1d-5aeaac0784c9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:22:33.919517 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:33.919418    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-kube-api-access-vdckp" pod "2ad11a1a-809c-482b-ba1d-5aeaac0784c9" (UID: "2ad11a1a-809c-482b-ba1d-5aeaac0784c9"). InnerVolumeSpecName "kube-api-access-vdckp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:22:34.018781 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.018717    5781 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-webhook-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:34.018781 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.018742    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdckp\" (UniqueName: \"kubernetes.io/projected/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-kube-api-access-vdckp\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:34.018781 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.018752    5781 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2ad11a1a-809c-482b-ba1d-5aeaac0784c9-metrics-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:22:34.513054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.513020    5781 generic.go:356] "Generic (PLEG): container finished" podID="2ad11a1a-809c-482b-ba1d-5aeaac0784c9" containerID="a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04" exitCode=0
Oct 30 13:22:34.513523 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.513077    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5444d7fdb7-whr85"
Oct 30 13:22:34.513523 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.513103    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-whr85" event={"ID":"2ad11a1a-809c-482b-ba1d-5aeaac0784c9","Type":"ContainerDied","Data":"a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04"}
Oct 30 13:22:34.513523 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.513181    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5444d7fdb7-whr85" event={"ID":"2ad11a1a-809c-482b-ba1d-5aeaac0784c9","Type":"ContainerDied","Data":"9d939873dc538fa6c32e6a1378e464d1792602143a39789feba01618cde2a087"}
Oct 30 13:22:34.513523 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.513210    5781 scope.go:122] "RemoveContainer" containerID="a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04"
Oct 30 13:22:34.522960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.522943    5781 scope.go:122] "RemoveContainer" containerID="a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04"
Oct 30 13:22:34.523256 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:22:34.523231    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04\": container with ID starting with a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04 not found: ID does not exist" containerID="a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04"
Oct 30 13:22:34.523372 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.523269    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04"} err="failed to get container status \"a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04\": rpc error: code = NotFound desc = could not find container \"a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04\": container with ID starting with a99fea056d8629d6b31767c8716fcdabd2b84e6589b2885dc3f87fff37bade04 not found: ID does not exist"
Oct 30 13:22:34.566098 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.566072    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-whr85"]
Oct 30 13:22:34.578860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.578835    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5444d7fdb7-whr85"]
Oct 30 13:22:34.839665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:34.839591    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ad11a1a-809c-482b-ba1d-5aeaac0784c9" path="/var/lib/kubelet/pods/2ad11a1a-809c-482b-ba1d-5aeaac0784c9/volumes"
Oct 30 13:22:42.043023 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:42.042988    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"]
Oct 30 13:22:42.054137 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:42.054105    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76e0a231-wrvbj"]
Oct 30 13:22:42.841008 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:42.840972    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dda450ba-2d78-475e-8070-7bf49a1b7e7e" path="/var/lib/kubelet/pods/dda450ba-2d78-475e-8070-7bf49a1b7e7e/volumes"
Oct 30 13:22:43.038597 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:43.038566    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:22:43.040343 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:43.040321    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:22:44.806804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:44.806775    5781 scope.go:122] "RemoveContainer" containerID="04b495d6e237e941cb99489277782d0dad781aa278e581d5317f28eaa2d2385c"
Oct 30 13:22:44.815698 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:44.815678    5781 scope.go:122] "RemoveContainer" containerID="93736a243db39b778ce165e9ddd52f5e1ac03b301ddd14d0ea1cceaaf9d05eb3"
Oct 30 13:22:44.823823 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:44.823803    5781 scope.go:122] "RemoveContainer" containerID="82072b1de625df191ae41adc8e131f768a7ecc7e287b14f34972263a73e81589"
Oct 30 13:22:44.831374 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:44.831355    5781 scope.go:122] "RemoveContainer" containerID="dce3d4fadca66f127beb0e31704b5fe30ff4b884427a40a8419b16072de04b45"
Oct 30 13:22:44.839413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:44.839396    5781 scope.go:122] "RemoveContainer" containerID="f6fdf15d3f7f0e34e53cf1a9162735ed877ec827fbd0dfe64e698b865efecfdb"
Oct 30 13:22:44.846748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:44.846723    5781 scope.go:122] "RemoveContainer" containerID="f59ced293b556cc8950b111af5f48569839ce7584526e12ce85bec074ceb4eb9"
Oct 30 13:22:51.562605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:51.562564    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:22:55.696591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:22:55.696555    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:23:01.994228 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:23:01.994187    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:23:14.038881 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:14.038843    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"]
Oct 30 13:23:14.049480 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:14.049457    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1ffa313c-dfvmf"]
Oct 30 13:23:14.840677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:14.840638    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="321ca0c6-0be9-4cf6-bad5-d9b91ed2096f" path="/var/lib/kubelet/pods/321ca0c6-0be9-4cf6-bad5-d9b91ed2096f/volumes"
Oct 30 13:23:24.035531 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:24.035456    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"]
Oct 30 13:23:24.046676 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:24.046644    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b8e408f6-qsqhp"]
Oct 30 13:23:24.839506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:24.839474    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf381d3f-d08f-43bd-973b-71c63ba88279" path="/var/lib/kubelet/pods/cf381d3f-d08f-43bd-973b-71c63ba88279/volumes"
Oct 30 13:23:31.970311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:31.967638    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:23:32.006275 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:23:32.006243    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:23:32.071962 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:32.071928    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"]
Oct 30 13:23:32.080709 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:32.080680    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-04d0ccf4-flksz"]
Oct 30 13:23:32.234166 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:32.234089    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:23:32.893006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:32.892972    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7179a4d-bc50-49c9-b605-27be4224c533" path="/var/lib/kubelet/pods/c7179a4d-bc50-49c9-b605-27be4224c533/volumes"
Oct 30 13:23:44.903666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:44.903635    5781 scope.go:122] "RemoveContainer" containerID="d100b2324988ea600e867cfaec50c7ba7e68fa0dbf344fe7825f4a8000f02822"
Oct 30 13:23:44.912652 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:44.912633    5781 scope.go:122] "RemoveContainer" containerID="aa8ce2bf86690f38abe376a56aafbfecaf6a180e1441f499dd9e014a6dbd00ba"
Oct 30 13:23:44.920322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:44.920298    5781 scope.go:122] "RemoveContainer" containerID="45259e454e4104b44b5fc7fd2d4d28b62538da5859ee9bbd77a16d1fdcfee17e"
Oct 30 13:23:47.803459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:47.803421    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:23:51.035078 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:51.035034    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"]
Oct 30 13:23:51.046258 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:51.046236    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9147fa74-tlbxq"]
Oct 30 13:23:51.755843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:51.755809    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:23:52.843168 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:52.843125    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f824fe6-a082-49b5-8168-9d97134c3c4c" path="/var/lib/kubelet/pods/9f824fe6-a082-49b5-8168-9d97134c3c4c/volumes"
Oct 30 13:23:56.042795 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:56.042764    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"]
Oct 30 13:23:56.053709 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:56.053683    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d2e932a-jcjvf"]
Oct 30 13:23:56.841112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:23:56.841078    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac0313fc-6f80-42fe-a4e7-7b339f88a597" path="/var/lib/kubelet/pods/ac0313fc-6f80-42fe-a4e7-7b339f88a597/volumes"
Oct 30 13:24:01.996150 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:24:01.996110    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:24:10.044164 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:10.044127    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"]
Oct 30 13:24:10.056318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:10.056279    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f0ef9c8e-lnr9x"]
Oct 30 13:24:10.839998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:10.839964    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab4d4ab3-8a37-4774-a85e-4108eb8354eb" path="/var/lib/kubelet/pods/ab4d4ab3-8a37-4774-a85e-4108eb8354eb/volumes"
Oct 30 13:24:12.033834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:12.033803    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"]
Oct 30 13:24:12.044396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:12.044364    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-61ad8f8b-mqszs"]
Oct 30 13:24:12.841071 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:12.841039    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43" path="/var/lib/kubelet/pods/059dcfe8-67c7-4d33-9b6b-f49cc4dd9b43/volumes"
Oct 30 13:24:16.032985 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:16.032946    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"]
Oct 30 13:24:16.044305 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:16.044264    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0f00f912-ptrn4"]
Oct 30 13:24:16.222252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:16.222207    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:24:16.841098 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:16.841061    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b92d8ba7-d527-4258-b054-18a1f5d112e5" path="/var/lib/kubelet/pods/b92d8ba7-d527-4258-b054-18a1f5d112e5/volumes"
Oct 30 13:24:22.068207 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:22.068173    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"]
Oct 30 13:24:22.079437 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:22.079404    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b06036fa-46vqf"]
Oct 30 13:24:22.840968 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:22.840929    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63254bd2-cb27-4bd6-a4c4-5562a9d6262c" path="/var/lib/kubelet/pods/63254bd2-cb27-4bd6-a4c4-5562a9d6262c/volumes"
Oct 30 13:24:26.057741 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:26.057698    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"]
Oct 30 13:24:26.068950 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:26.068925    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-be6b5d15-dlg56"]
Oct 30 13:24:26.840626 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:26.840584    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc4cdbe5-257b-4f88-a491-21745923343b" path="/var/lib/kubelet/pods/dc4cdbe5-257b-4f88-a491-21745923343b/volumes"
Oct 30 13:24:31.995642 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:24:31.995598    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:24:32.363394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:32.363326    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:24:38.040903 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:38.040863    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"]
Oct 30 13:24:38.052025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:38.051998    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a11dbc35-pksgl"]
Oct 30 13:24:38.840411 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:38.840378    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd44d6d9-e84d-4823-adbe-51513dffde59" path="/var/lib/kubelet/pods/cd44d6d9-e84d-4823-adbe-51513dffde59/volumes"
Oct 30 13:24:44.950704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:44.950671    5781 scope.go:122] "RemoveContainer" containerID="37669ae876e7e7c01f29e196c5a0907b71c92621eab56b07990d91d65310989c"
Oct 30 13:24:44.959649 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:44.959628    5781 scope.go:122] "RemoveContainer" containerID="abd5e062c392d028498e596f06b2088d947b402c619fe9f22ddde39d1f224a56"
Oct 30 13:24:44.967437 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:44.967418    5781 scope.go:122] "RemoveContainer" containerID="fcc9d30fb8ea2059833338c8096ad72ca47aeb156272008ebc3223a749353c30"
Oct 30 13:24:53.031922 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:53.031720    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"]
Oct 30 13:24:53.042846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:53.042819    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f8d5f7cc-ssnvg"]
Oct 30 13:24:54.840109 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:54.840077    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3e2b109-3318-4037-8d35-ad4aea11ce83" path="/var/lib/kubelet/pods/e3e2b109-3318-4037-8d35-ad4aea11ce83/volumes"
Oct 30 13:24:56.384167 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:56.384135    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:24:56.524136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:24:56.524108    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:25:01.842440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:01.842408    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:25:01.994271 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:25:01.994239    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:25:16.040618 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:16.040582    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"]
Oct 30 13:25:16.058713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:16.058687    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fb15dd07-dlhzd"]
Oct 30 13:25:16.839382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:16.839352    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3cb4f7f2-0b7d-4aec-a49f-0811a1b16021" path="/var/lib/kubelet/pods/3cb4f7f2-0b7d-4aec-a49f-0811a1b16021/volumes"
Oct 30 13:25:31.994068 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:25:31.994032    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:25:32.456228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:32.456195    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:25:35.702502 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:35.702458    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"]
Oct 30 13:25:35.702978 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:35.702715    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw" podUID="f914ca73-c724-4f0f-b59a-0a9c7fd8143d" containerName="discovery" containerID="cri-o://136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484" gracePeriod=30
Oct 30 13:25:35.984339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:35.984312    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:25:36.109148 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.109116    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-local-certs\") pod \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") "
Oct 30 13:25:36.109364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.109184    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-ca-configmap\") pod \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") "
Oct 30 13:25:36.109364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.109223    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-kube-api-access-p54jv\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-kube-api-access-p54jv\") pod \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") "
Oct 30 13:25:36.109364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.109260    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-token\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-token\") pod \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") "
Oct 30 13:25:36.109364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.109335    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-kubeconfig\") pod \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") "
Oct 30 13:25:36.109621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.109376    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-cacerts\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-cacerts\") pod \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") "
Oct 30 13:25:36.109621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.109401    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-dns-cert\") pod \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\" (UID: \"f914ca73-c724-4f0f-b59a-0a9c7fd8143d\") "
Oct 30 13:25:36.109621 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.109602    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-ca-configmap" pod "f914ca73-c724-4f0f-b59a-0a9c7fd8143d" (UID: "f914ca73-c724-4f0f-b59a-0a9c7fd8143d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:25:36.109901 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.109881    5781 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-ca-configmap\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:36.111442 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.111417    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-local-certs" pod "f914ca73-c724-4f0f-b59a-0a9c7fd8143d" (UID: "f914ca73-c724-4f0f-b59a-0a9c7fd8143d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:25:36.111761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.111738    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-kubeconfig" pod "f914ca73-c724-4f0f-b59a-0a9c7fd8143d" (UID: "f914ca73-c724-4f0f-b59a-0a9c7fd8143d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:25:36.111933 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.111907    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-cacerts" pod "f914ca73-c724-4f0f-b59a-0a9c7fd8143d" (UID: "f914ca73-c724-4f0f-b59a-0a9c7fd8143d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:25:36.112025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.111934    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-kube-api-access-p54jv" pod "f914ca73-c724-4f0f-b59a-0a9c7fd8143d" (UID: "f914ca73-c724-4f0f-b59a-0a9c7fd8143d"). InnerVolumeSpecName "kube-api-access-p54jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:25:36.112025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.111954    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-dns-cert" pod "f914ca73-c724-4f0f-b59a-0a9c7fd8143d" (UID: "f914ca73-c724-4f0f-b59a-0a9c7fd8143d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:25:36.112025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.111959    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-token" pod "f914ca73-c724-4f0f-b59a-0a9c7fd8143d" (UID: "f914ca73-c724-4f0f-b59a-0a9c7fd8143d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:25:36.211091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.211010    5781 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-cacerts\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:36.211091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.211039    5781 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-csr-dns-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:36.211091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.211050    5781 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-local-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:36.211091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.211059    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p54jv\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-kube-api-access-p54jv\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:36.211091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.211067    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:36.211091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.211076    5781 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f914ca73-c724-4f0f-b59a-0a9c7fd8143d-istio-kubeconfig\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:36.609528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.609437    5781 generic.go:356] "Generic (PLEG): container finished" podID="f914ca73-c724-4f0f-b59a-0a9c7fd8143d" containerID="136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484" exitCode=0
Oct 30 13:25:36.609528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.609504    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"
Oct 30 13:25:36.609528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.609516    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw" event={"ID":"f914ca73-c724-4f0f-b59a-0a9c7fd8143d","Type":"ContainerDied","Data":"136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484"}
Oct 30 13:25:36.609821 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.609587    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw" event={"ID":"f914ca73-c724-4f0f-b59a-0a9c7fd8143d","Type":"ContainerDied","Data":"c45c1826cdd268291fd91229b0698ff119f08d86502b3fcd26ddf62c26aae68d"}
Oct 30 13:25:36.609821 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.609617    5781 scope.go:122] "RemoveContainer" containerID="136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484"
Oct 30 13:25:36.619729 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.619681    5781 scope.go:122] "RemoveContainer" containerID="136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484"
Oct 30 13:25:36.620063 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:25:36.619999    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484\": container with ID starting with 136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484 not found: ID does not exist" containerID="136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484"
Oct 30 13:25:36.620063 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.620042    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484"} err="failed to get container status \"136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484\": rpc error: code = NotFound desc = could not find container \"136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484\": container with ID starting with 136bc27417354b7aa9678bb0a725747e8cb0355d54407758d872dff6ac5f5484 not found: ID does not exist"
Oct 30 13:25:36.677226 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.677189    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"]
Oct 30 13:25:36.688420 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.688390    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-p28dw"]
Oct 30 13:25:36.840761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:36.840724    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f914ca73-c724-4f0f-b59a-0a9c7fd8143d" path="/var/lib/kubelet/pods/f914ca73-c724-4f0f-b59a-0a9c7fd8143d/volumes"
Oct 30 13:25:37.066749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:37.066712    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:25:43.711983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.711939    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"]
Oct 30 13:25:43.712662 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.712638    5781 cpu_manager.go:414] "containerMap: removing container" podUID="f914ca73-c724-4f0f-b59a-0a9c7fd8143d" containerName="discovery"
Oct 30 13:25:43.712843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.712817    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f914ca73-c724-4f0f-b59a-0a9c7fd8143d" containerName="discovery"
Oct 30 13:25:43.712956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.712856    5781 cpu_manager.go:414] "containerMap: removing container" podUID="2ad11a1a-809c-482b-ba1d-5aeaac0784c9" containerName="manager"
Oct 30 13:25:43.712956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.712866    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ad11a1a-809c-482b-ba1d-5aeaac0784c9" containerName="manager"
Oct 30 13:25:43.713048 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.713019    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="f914ca73-c724-4f0f-b59a-0a9c7fd8143d" containerName="discovery"
Oct 30 13:25:43.713048 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.713033    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ad11a1a-809c-482b-ba1d-5aeaac0784c9" containerName="manager"
Oct 30 13:25:43.718332 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.718312    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:43.742546 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.742513    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-fbq8w\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:43.742715 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.742613    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df4l8\" (UniqueName: \"kubernetes.io/projected/cea85225-e1cb-465e-9b58-30a1c66576b5-kube-api-access-df4l8\") pod \"maas-api-revoke-sub-a4aef7d7-fbq8w\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:43.742715 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.742681    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-fbq8w\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:43.744357 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.744333    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:25:43.744532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.744373    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:25:43.746551 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.746529    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"]
Oct 30 13:25:43.843628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.843594    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-fbq8w\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:43.843828 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.843654    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df4l8\" (UniqueName: \"kubernetes.io/projected/cea85225-e1cb-465e-9b58-30a1c66576b5-kube-api-access-df4l8\") pod \"maas-api-revoke-sub-a4aef7d7-fbq8w\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:43.843828 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.843695    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-fbq8w\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:43.844318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.844279    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a4aef7d7-fbq8w\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:43.845909 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.845885    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a4aef7d7-fbq8w\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:43.866732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:43.866703    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-df4l8\" (UniqueName: \"kubernetes.io/projected/cea85225-e1cb-465e-9b58-30a1c66576b5-kube-api-access-df4l8\") pod \"maas-api-revoke-sub-a4aef7d7-fbq8w\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:44.030801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:44.030712    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:44.329337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:44.329234    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"]
Oct 30 13:25:44.333344 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:25:44.333313    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea85225_e1cb_465e_9b58_30a1c66576b5.slice/crio-9cbf87c8f8c5ed3186b93d02e8ac28161741a776e1e38d60d527d365b2dcd5e6 WatchSource:0}: Error finding container 9cbf87c8f8c5ed3186b93d02e8ac28161741a776e1e38d60d527d365b2dcd5e6: Status 404 returned error can't find the container with id 9cbf87c8f8c5ed3186b93d02e8ac28161741a776e1e38d60d527d365b2dcd5e6
Oct 30 13:25:44.689268 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:44.689229    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w" event={"ID":"cea85225-e1cb-465e-9b58-30a1c66576b5","Type":"ContainerStarted","Data":"eb4901a608ad25a1d0e8366b370b1eb5e78952cc43bb301ce13b6c1922036192"}
Oct 30 13:25:44.689441 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:44.689340    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w" event={"ID":"cea85225-e1cb-465e-9b58-30a1c66576b5","Type":"ContainerStarted","Data":"9cbf87c8f8c5ed3186b93d02e8ac28161741a776e1e38d60d527d365b2dcd5e6"}
Oct 30 13:25:44.729950 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:44.729901    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w" podStartSLOduration=1.72988814 podStartE2EDuration="1.72988814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:25:44.728881737 +0000 UTC m=+1682.026099289" watchObservedRunningTime="2026-10-30 13:25:44.72988814 +0000 UTC m=+1682.027105682"
Oct 30 13:25:44.997917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:44.997882    5781 scope.go:122] "RemoveContainer" containerID="5eda19b1106a3403b280cbd9d81fe0da8e818417d4b2b5abdd6b0425b99c4b7e"
Oct 30 13:25:45.007945 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.007926    5781 scope.go:122] "RemoveContainer" containerID="c7876ff084237f8c2d070efaf92ff11ec8575fd7cc5183da4eb1925169b52784"
Oct 30 13:25:45.016318 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.016282    5781 scope.go:122] "RemoveContainer" containerID="14abef82f64b11732e68669de146fdbe9e6b9894c74d9d05e94ea88d569e9461"
Oct 30 13:25:45.023877 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.023858    5781 scope.go:122] "RemoveContainer" containerID="29f4a742694ea3ff226762c4b1a29961c029afdb493c02848f7d4bfd1d6cffd0"
Oct 30 13:25:45.031324 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.031301    5781 scope.go:122] "RemoveContainer" containerID="71b05b87023678cd519198c124bfbf4c647e786b1e4d606b7e70fb5002ed6921"
Oct 30 13:25:45.038794 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.038638    5781 scope.go:122] "RemoveContainer" containerID="66d4bb58e6b25420f891d3647ebe21c65c202183f1bac02cf78d6cadb8b19511"
Oct 30 13:25:45.045846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.045829    5781 scope.go:122] "RemoveContainer" containerID="18ad39283c5ee803a22e6790831da42f0459a01801f9d89e8246a04b45a49672"
Oct 30 13:25:45.053674 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.053658    5781 scope.go:122] "RemoveContainer" containerID="a82e83669973de34cc7d76a7be25b0453fda74fc763f36b4aeec2367e3e48623"
Oct 30 13:25:45.061144 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.061115    5781 scope.go:122] "RemoveContainer" containerID="a6cba356a9865f2241c0798f6ca09ad83be3185bd0d2501be180ba83e5d84ed9"
Oct 30 13:25:45.695963 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.695932    5781 generic.go:356] "Generic (PLEG): container finished" podID="cea85225-e1cb-465e-9b58-30a1c66576b5" containerID="eb4901a608ad25a1d0e8366b370b1eb5e78952cc43bb301ce13b6c1922036192" exitCode=0
Oct 30 13:25:45.696182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:45.696026    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w" event={"ID":"cea85225-e1cb-465e-9b58-30a1c66576b5","Type":"ContainerDied","Data":"eb4901a608ad25a1d0e8366b370b1eb5e78952cc43bb301ce13b6c1922036192"}
Oct 30 13:25:46.832392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.832360    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:46.872392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.872363    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cea85225-e1cb-465e-9b58-30a1c66576b5-kube-api-access-df4l8\" (UniqueName: \"kubernetes.io/projected/cea85225-e1cb-465e-9b58-30a1c66576b5-kube-api-access-df4l8\") pod \"cea85225-e1cb-465e-9b58-30a1c66576b5\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") "
Oct 30 13:25:46.872502 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.872451    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-openshift-ca\") pod \"cea85225-e1cb-465e-9b58-30a1c66576b5\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") "
Oct 30 13:25:46.872542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.872513    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-xks-ca\") pod \"cea85225-e1cb-465e-9b58-30a1c66576b5\" (UID: \"cea85225-e1cb-465e-9b58-30a1c66576b5\") "
Oct 30 13:25:46.872809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.872783    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-openshift-ca" pod "cea85225-e1cb-465e-9b58-30a1c66576b5" (UID: "cea85225-e1cb-465e-9b58-30a1c66576b5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:25:46.874581 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.874556    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea85225-e1cb-465e-9b58-30a1c66576b5-kube-api-access-df4l8" pod "cea85225-e1cb-465e-9b58-30a1c66576b5" (UID: "cea85225-e1cb-465e-9b58-30a1c66576b5"). InnerVolumeSpecName "kube-api-access-df4l8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:25:46.874754 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.874719    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-xks-ca" pod "cea85225-e1cb-465e-9b58-30a1c66576b5" (UID: "cea85225-e1cb-465e-9b58-30a1c66576b5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:25:46.973321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.973254    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-df4l8\" (UniqueName: \"kubernetes.io/projected/cea85225-e1cb-465e-9b58-30a1c66576b5-kube-api-access-df4l8\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:46.973321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.973275    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:46.973321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:46.973307    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cea85225-e1cb-465e-9b58-30a1c66576b5-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:25:47.707850 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:47.707821    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"
Oct 30 13:25:47.708018 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:47.707864    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w" event={"ID":"cea85225-e1cb-465e-9b58-30a1c66576b5","Type":"ContainerDied","Data":"9cbf87c8f8c5ed3186b93d02e8ac28161741a776e1e38d60d527d365b2dcd5e6"}
Oct 30 13:25:47.708018 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:47.707914    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9cbf87c8f8c5ed3186b93d02e8ac28161741a776e1e38d60d527d365b2dcd5e6"
Oct 30 13:25:59.034955 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:59.034923    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"]
Oct 30 13:25:59.046083 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:25:59.046058    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8951bdfd-qbkpp"]
Oct 30 13:26:00.857311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:00.846198    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0727e035-c05f-4655-a822-6057e6d2ddb0" path="/var/lib/kubelet/pods/0727e035-c05f-4655-a822-6057e6d2ddb0/volumes"
Oct 30 13:26:02.039196 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:26:02.039134    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:26:04.020489 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.020458    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"]
Oct 30 13:26:04.021103 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.021081    5781 cpu_manager.go:414] "containerMap: removing container" podUID="cea85225-e1cb-465e-9b58-30a1c66576b5" containerName="revoke-keys"
Oct 30 13:26:04.021192 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.021104    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cea85225-e1cb-465e-9b58-30a1c66576b5" containerName="revoke-keys"
Oct 30 13:26:04.021269 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.021254    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="cea85225-e1cb-465e-9b58-30a1c66576b5" containerName="revoke-keys"
Oct 30 13:26:04.024436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.024415    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.039228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.039208    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:26:04.050902 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.050881    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:26:04.052570 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.052549    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"]
Oct 30 13:26:04.166024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.165993    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-bg29r\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.166220 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.166053    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-bg29r\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.166220 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.166179    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqmgf\" (UniqueName: \"kubernetes.io/projected/32b7861f-eabe-4b9f-be4d-d77194df97e9-kube-api-access-pqmgf\") pod \"maas-api-revoke-sub-79805ca9-bg29r\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.266929 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.266893    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-bg29r\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.267085 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.266957    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-bg29r\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.267085 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.266992    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqmgf\" (UniqueName: \"kubernetes.io/projected/32b7861f-eabe-4b9f-be4d-d77194df97e9-kube-api-access-pqmgf\") pod \"maas-api-revoke-sub-79805ca9-bg29r\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.267704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.267687    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-79805ca9-bg29r\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.269105 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.269079    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-79805ca9-bg29r\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.290058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.290008    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqmgf\" (UniqueName: \"kubernetes.io/projected/32b7861f-eabe-4b9f-be4d-d77194df97e9-kube-api-access-pqmgf\") pod \"maas-api-revoke-sub-79805ca9-bg29r\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.337092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.337073    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:04.635256 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.635181    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"]
Oct 30 13:26:04.639119 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:26:04.639090    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b7861f_eabe_4b9f_be4d_d77194df97e9.slice/crio-0412fec4b69f16aaefd4ed0326a7009ee50f3a0f9c822f03714287ee223c0362 WatchSource:0}: Error finding container 0412fec4b69f16aaefd4ed0326a7009ee50f3a0f9c822f03714287ee223c0362: Status 404 returned error can't find the container with id 0412fec4b69f16aaefd4ed0326a7009ee50f3a0f9c822f03714287ee223c0362
Oct 30 13:26:04.809778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:04.809745    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r" event={"ID":"32b7861f-eabe-4b9f-be4d-d77194df97e9","Type":"ContainerStarted","Data":"0412fec4b69f16aaefd4ed0326a7009ee50f3a0f9c822f03714287ee223c0362"}
Oct 30 13:26:05.816406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:05.816315    5781 generic.go:356] "Generic (PLEG): container finished" podID="32b7861f-eabe-4b9f-be4d-d77194df97e9" containerID="14e5cce602fb19b9ddda57c7880b80907bc48de6333e1eb42d3b87dc50b0ab7c" exitCode=0
Oct 30 13:26:05.816406 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:05.816368    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r" event={"ID":"32b7861f-eabe-4b9f-be4d-d77194df97e9","Type":"ContainerDied","Data":"14e5cce602fb19b9ddda57c7880b80907bc48de6333e1eb42d3b87dc50b0ab7c"}
Oct 30 13:26:06.950604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:06.950581    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:07.051169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.051139    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"]
Oct 30 13:26:07.074580 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.074549    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-fce879a9-7f7js"]
Oct 30 13:26:07.094146 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.094121    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-openshift-ca\") pod \"32b7861f-eabe-4b9f-be4d-d77194df97e9\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") "
Oct 30 13:26:07.094313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.094188    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32b7861f-eabe-4b9f-be4d-d77194df97e9-kube-api-access-pqmgf\" (UniqueName: \"kubernetes.io/projected/32b7861f-eabe-4b9f-be4d-d77194df97e9-kube-api-access-pqmgf\") pod \"32b7861f-eabe-4b9f-be4d-d77194df97e9\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") "
Oct 30 13:26:07.094382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.094368    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-xks-ca\") pod \"32b7861f-eabe-4b9f-be4d-d77194df97e9\" (UID: \"32b7861f-eabe-4b9f-be4d-d77194df97e9\") "
Oct 30 13:26:07.094574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.094551    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-openshift-ca" pod "32b7861f-eabe-4b9f-be4d-d77194df97e9" (UID: "32b7861f-eabe-4b9f-be4d-d77194df97e9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:26:07.094753 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.094736    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:07.096197 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.096174    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b7861f-eabe-4b9f-be4d-d77194df97e9-kube-api-access-pqmgf" pod "32b7861f-eabe-4b9f-be4d-d77194df97e9" (UID: "32b7861f-eabe-4b9f-be4d-d77194df97e9"). InnerVolumeSpecName "kube-api-access-pqmgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:26:07.096325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.096305    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-xks-ca" pod "32b7861f-eabe-4b9f-be4d-d77194df97e9" (UID: "32b7861f-eabe-4b9f-be4d-d77194df97e9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:26:07.195926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.195903    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32b7861f-eabe-4b9f-be4d-d77194df97e9-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:07.195926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.195924    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqmgf\" (UniqueName: \"kubernetes.io/projected/32b7861f-eabe-4b9f-be4d-d77194df97e9-kube-api-access-pqmgf\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:07.829149 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.829096    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r" event={"ID":"32b7861f-eabe-4b9f-be4d-d77194df97e9","Type":"ContainerDied","Data":"0412fec4b69f16aaefd4ed0326a7009ee50f3a0f9c822f03714287ee223c0362"}
Oct 30 13:26:07.829370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.829211    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0412fec4b69f16aaefd4ed0326a7009ee50f3a0f9c822f03714287ee223c0362"
Oct 30 13:26:07.829370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:07.829315    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"
Oct 30 13:26:08.839899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:08.839866    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75e6e1a7-832b-4a9f-8683-c66c5876c1f7" path="/var/lib/kubelet/pods/75e6e1a7-832b-4a9f-8683-c66c5876c1f7/volumes"
Oct 30 13:26:18.910452 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:18.910417    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:26:18.910852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:18.910461    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:26:18.949309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:18.948389    5781 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:26:18.949309 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:18.948485    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:26:21.949986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:21.949778    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"]
Oct 30 13:26:21.950440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:21.950420    5781 cpu_manager.go:414] "containerMap: removing container" podUID="32b7861f-eabe-4b9f-be4d-d77194df97e9" containerName="revoke-keys"
Oct 30 13:26:21.950440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:21.950440    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32b7861f-eabe-4b9f-be4d-d77194df97e9" containerName="revoke-keys"
Oct 30 13:26:21.950574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:21.950530    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="32b7861f-eabe-4b9f-be4d-d77194df97e9" containerName="revoke-keys"
Oct 30 13:26:21.954903 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:21.954888    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:21.969167 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:21.969146    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:26:21.980551 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:21.980527    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:26:21.982528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:21.982506    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"]
Oct 30 13:26:22.139331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.139301    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-pjh58\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.139485 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.139400    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsd88\" (UniqueName: \"kubernetes.io/projected/37001bb8-a1e3-4a93-a2dc-7817a10099d4-kube-api-access-nsd88\") pod \"maas-api-revoke-sub-0b7e8edb-pjh58\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.139485 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.139445    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-pjh58\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.240639 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.240562    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-pjh58\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.240639 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.240635    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsd88\" (UniqueName: \"kubernetes.io/projected/37001bb8-a1e3-4a93-a2dc-7817a10099d4-kube-api-access-nsd88\") pod \"maas-api-revoke-sub-0b7e8edb-pjh58\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.240832 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.240674    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-pjh58\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.241351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.241329    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b7e8edb-pjh58\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.242864 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.242841    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b7e8edb-pjh58\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.263941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.263919    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsd88\" (UniqueName: \"kubernetes.io/projected/37001bb8-a1e3-4a93-a2dc-7817a10099d4-kube-api-access-nsd88\") pod \"maas-api-revoke-sub-0b7e8edb-pjh58\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.267739 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.267718    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:22.564598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.564521    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"]
Oct 30 13:26:22.569297 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:26:22.569259    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37001bb8_a1e3_4a93_a2dc_7817a10099d4.slice/crio-c357641d4571bca1fd30d149c2c52b28e3b15a4973aa70c7494506187fabb8ff WatchSource:0}: Error finding container c357641d4571bca1fd30d149c2c52b28e3b15a4973aa70c7494506187fabb8ff: Status 404 returned error can't find the container with id c357641d4571bca1fd30d149c2c52b28e3b15a4973aa70c7494506187fabb8ff
Oct 30 13:26:22.908814 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.908746    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58" event={"ID":"37001bb8-a1e3-4a93-a2dc-7817a10099d4","Type":"ContainerStarted","Data":"204c92deda67f6b45f93559fa85bc342f96c8227bd63f83965f60f89658ecda1"}
Oct 30 13:26:22.908814 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.908803    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58" event={"ID":"37001bb8-a1e3-4a93-a2dc-7817a10099d4","Type":"ContainerStarted","Data":"c357641d4571bca1fd30d149c2c52b28e3b15a4973aa70c7494506187fabb8ff"}
Oct 30 13:26:22.948476 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:22.948433    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58" podStartSLOduration=1.9484199549999999 podStartE2EDuration="1.948419955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:26:22.947488593 +0000 UTC m=+1720.244706134" watchObservedRunningTime="2026-10-30 13:26:22.948419955 +0000 UTC m=+1720.245637496"
Oct 30 13:26:23.915672 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:23.915638    5781 generic.go:356] "Generic (PLEG): container finished" podID="37001bb8-a1e3-4a93-a2dc-7817a10099d4" containerID="204c92deda67f6b45f93559fa85bc342f96c8227bd63f83965f60f89658ecda1" exitCode=0
Oct 30 13:26:23.916091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:23.915708    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58" event={"ID":"37001bb8-a1e3-4a93-a2dc-7817a10099d4","Type":"ContainerDied","Data":"204c92deda67f6b45f93559fa85bc342f96c8227bd63f83965f60f89658ecda1"}
Oct 30 13:26:25.052433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.052406    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:25.166620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.166591    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-xks-ca\") pod \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") "
Oct 30 13:26:25.166780 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.166631    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-openshift-ca\") pod \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") "
Oct 30 13:26:25.166780 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.166665    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37001bb8-a1e3-4a93-a2dc-7817a10099d4-kube-api-access-nsd88\" (UniqueName: \"kubernetes.io/projected/37001bb8-a1e3-4a93-a2dc-7817a10099d4-kube-api-access-nsd88\") pod \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\" (UID: \"37001bb8-a1e3-4a93-a2dc-7817a10099d4\") "
Oct 30 13:26:25.166964 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.166940    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-openshift-ca" pod "37001bb8-a1e3-4a93-a2dc-7817a10099d4" (UID: "37001bb8-a1e3-4a93-a2dc-7817a10099d4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:26:25.168701 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.168675    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37001bb8-a1e3-4a93-a2dc-7817a10099d4-kube-api-access-nsd88" pod "37001bb8-a1e3-4a93-a2dc-7817a10099d4" (UID: "37001bb8-a1e3-4a93-a2dc-7817a10099d4"). InnerVolumeSpecName "kube-api-access-nsd88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:26:25.168775 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.168680    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-xks-ca" pod "37001bb8-a1e3-4a93-a2dc-7817a10099d4" (UID: "37001bb8-a1e3-4a93-a2dc-7817a10099d4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:26:25.268045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.268022    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:25.268045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.268044    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/37001bb8-a1e3-4a93-a2dc-7817a10099d4-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:25.268167 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.268053    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsd88\" (UniqueName: \"kubernetes.io/projected/37001bb8-a1e3-4a93-a2dc-7817a10099d4-kube-api-access-nsd88\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:25.926229 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.926190    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58" event={"ID":"37001bb8-a1e3-4a93-a2dc-7817a10099d4","Type":"ContainerDied","Data":"c357641d4571bca1fd30d149c2c52b28e3b15a4973aa70c7494506187fabb8ff"}
Oct 30 13:26:25.926385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.926241    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c357641d4571bca1fd30d149c2c52b28e3b15a4973aa70c7494506187fabb8ff"
Oct 30 13:26:25.926385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:25.926207    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"
Oct 30 13:26:31.994081 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:26:31.994043    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:26:32.621007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.620972    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:26:32.720183 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.720151    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"]
Oct 30 13:26:32.720678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.720660    5781 cpu_manager.go:414] "containerMap: removing container" podUID="37001bb8-a1e3-4a93-a2dc-7817a10099d4" containerName="revoke-keys"
Oct 30 13:26:32.720749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.720679    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37001bb8-a1e3-4a93-a2dc-7817a10099d4" containerName="revoke-keys"
Oct 30 13:26:32.720792 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.720776    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="37001bb8-a1e3-4a93-a2dc-7817a10099d4" containerName="revoke-keys"
Oct 30 13:26:32.723713 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.723695    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:32.752735 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.752706    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:26:32.752735 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.752728    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:26:32.755546 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.755518    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"]
Oct 30 13:26:32.827663 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.827635    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-tbvcx\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:32.827819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.827672    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhpc4\" (UniqueName: \"kubernetes.io/projected/0183e57f-0165-437e-9c43-059fb1890e2b-kube-api-access-xhpc4\") pod \"maas-api-revoke-sub-3d4156f1-tbvcx\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:32.827819 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.827690    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-tbvcx\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:32.928782 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.928754    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-tbvcx\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:32.928930 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.928810    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhpc4\" (UniqueName: \"kubernetes.io/projected/0183e57f-0165-437e-9c43-059fb1890e2b-kube-api-access-xhpc4\") pod \"maas-api-revoke-sub-3d4156f1-tbvcx\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:32.928930 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.928841    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-tbvcx\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:32.929394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.929372    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-tbvcx\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:32.931014 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.930995    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-tbvcx\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:32.951369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:32.951347    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhpc4\" (UniqueName: \"kubernetes.io/projected/0183e57f-0165-437e-9c43-059fb1890e2b-kube-api-access-xhpc4\") pod \"maas-api-revoke-sub-3d4156f1-tbvcx\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:33.039603 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:33.039582    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:33.337314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:33.337222    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"]
Oct 30 13:26:33.341467 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:26:33.341436    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0183e57f_0165_437e_9c43_059fb1890e2b.slice/crio-f3aff4faf8890ae08c09def23e6ae1e6c49f54c2ed75a7714f4d63c798a58042 WatchSource:0}: Error finding container f3aff4faf8890ae08c09def23e6ae1e6c49f54c2ed75a7714f4d63c798a58042: Status 404 returned error can't find the container with id f3aff4faf8890ae08c09def23e6ae1e6c49f54c2ed75a7714f4d63c798a58042
Oct 30 13:26:33.968511 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:33.968477    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx" event={"ID":"0183e57f-0165-437e-9c43-059fb1890e2b","Type":"ContainerStarted","Data":"e3853c5c932336cc244abb20c23d04107978c29441a42a6cd05a5e12a0d88a1f"}
Oct 30 13:26:33.968666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:33.968524    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx" event={"ID":"0183e57f-0165-437e-9c43-059fb1890e2b","Type":"ContainerStarted","Data":"f3aff4faf8890ae08c09def23e6ae1e6c49f54c2ed75a7714f4d63c798a58042"}
Oct 30 13:26:34.015916 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:34.015859    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx" podStartSLOduration=2.015845397 podStartE2EDuration="2.015845397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:26:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:26:34.013903186 +0000 UTC m=+1731.311120728" watchObservedRunningTime="2026-10-30 13:26:34.015845397 +0000 UTC m=+1731.313062938"
Oct 30 13:26:34.974906 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:34.974871    5781 generic.go:356] "Generic (PLEG): container finished" podID="0183e57f-0165-437e-9c43-059fb1890e2b" containerID="e3853c5c932336cc244abb20c23d04107978c29441a42a6cd05a5e12a0d88a1f" exitCode=0
Oct 30 13:26:34.975307 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:34.974954    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx" event={"ID":"0183e57f-0165-437e-9c43-059fb1890e2b","Type":"ContainerDied","Data":"e3853c5c932336cc244abb20c23d04107978c29441a42a6cd05a5e12a0d88a1f"}
Oct 30 13:26:35.045830 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:35.045643    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"]
Oct 30 13:26:35.054049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:35.054024    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-efb457e3-vrlst"]
Oct 30 13:26:36.106118 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.106095    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:36.159413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.159392    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-openshift-ca\") pod \"0183e57f-0165-437e-9c43-059fb1890e2b\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") "
Oct 30 13:26:36.159533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.159438    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0183e57f-0165-437e-9c43-059fb1890e2b-kube-api-access-xhpc4\" (UniqueName: \"kubernetes.io/projected/0183e57f-0165-437e-9c43-059fb1890e2b-kube-api-access-xhpc4\") pod \"0183e57f-0165-437e-9c43-059fb1890e2b\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") "
Oct 30 13:26:36.159533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.159469    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-xks-ca\") pod \"0183e57f-0165-437e-9c43-059fb1890e2b\" (UID: \"0183e57f-0165-437e-9c43-059fb1890e2b\") "
Oct 30 13:26:36.159820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.159788    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-openshift-ca" pod "0183e57f-0165-437e-9c43-059fb1890e2b" (UID: "0183e57f-0165-437e-9c43-059fb1890e2b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:26:36.161508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.161482    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-xks-ca" pod "0183e57f-0165-437e-9c43-059fb1890e2b" (UID: "0183e57f-0165-437e-9c43-059fb1890e2b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:26:36.161598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.161544    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0183e57f-0165-437e-9c43-059fb1890e2b-kube-api-access-xhpc4" pod "0183e57f-0165-437e-9c43-059fb1890e2b" (UID: "0183e57f-0165-437e-9c43-059fb1890e2b"). InnerVolumeSpecName "kube-api-access-xhpc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:26:36.260413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.260364    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:36.260413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.260383    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhpc4\" (UniqueName: \"kubernetes.io/projected/0183e57f-0165-437e-9c43-059fb1890e2b-kube-api-access-xhpc4\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:36.260413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.260392    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/0183e57f-0165-437e-9c43-059fb1890e2b-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:36.839708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.839667    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9bb522b8-462e-4b77-986d-5dd2ab247e09" path="/var/lib/kubelet/pods/9bb522b8-462e-4b77-986d-5dd2ab247e09/volumes"
Oct 30 13:26:36.985330 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.985302    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"
Oct 30 13:26:36.985446 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.985331    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx" event={"ID":"0183e57f-0165-437e-9c43-059fb1890e2b","Type":"ContainerDied","Data":"f3aff4faf8890ae08c09def23e6ae1e6c49f54c2ed75a7714f4d63c798a58042"}
Oct 30 13:26:36.985446 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:36.985373    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3aff4faf8890ae08c09def23e6ae1e6c49f54c2ed75a7714f4d63c798a58042"
Oct 30 13:26:37.215294 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:37.215266    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:26:45.114718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:45.114687    5781 scope.go:122] "RemoveContainer" containerID="750f9a65079e1a1c6c0a59380000b9a4eb89f7560d6ebbf4959be544593fbe02"
Oct 30 13:26:45.123188 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:45.123170    5781 scope.go:122] "RemoveContainer" containerID="c8a900b0ce9471ed6ac62732108316e5f088b002c5f6a9e44fa71563d75e4983"
Oct 30 13:26:54.230829 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.230791    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"]
Oct 30 13:26:54.242190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.242159    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-tbvcx"]
Oct 30 13:26:54.284998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.284971    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"]
Oct 30 13:26:54.285518 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.285499    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0183e57f-0165-437e-9c43-059fb1890e2b" containerName="revoke-keys"
Oct 30 13:26:54.285640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.285519    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0183e57f-0165-437e-9c43-059fb1890e2b" containerName="revoke-keys"
Oct 30 13:26:54.285698 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.285662    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0183e57f-0165-437e-9c43-059fb1890e2b" containerName="revoke-keys"
Oct 30 13:26:54.290525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.290505    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.304718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.304695    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:26:54.304913 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.304896    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:26:54.311540 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.311519    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"]
Oct 30 13:26:54.349341 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.349317    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-zhv7s\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.349472 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.349365    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmvp4\" (UniqueName: \"kubernetes.io/projected/8e2d089d-9c32-44be-ac77-4e9f415e46b0-kube-api-access-vmvp4\") pod \"maas-api-revoke-sub-3d4156f1-zhv7s\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.349533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.349496    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-zhv7s\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.450159 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.450134    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmvp4\" (UniqueName: \"kubernetes.io/projected/8e2d089d-9c32-44be-ac77-4e9f415e46b0-kube-api-access-vmvp4\") pod \"maas-api-revoke-sub-3d4156f1-zhv7s\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.450250 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.450226    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-zhv7s\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.450342 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.450326    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-zhv7s\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.450892 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.450871    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-zhv7s\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.452537 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.452520    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-zhv7s\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.473704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.473683    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmvp4\" (UniqueName: \"kubernetes.io/projected/8e2d089d-9c32-44be-ac77-4e9f415e46b0-kube-api-access-vmvp4\") pod \"maas-api-revoke-sub-3d4156f1-zhv7s\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.603704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.603652    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:54.840329 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.840279    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0183e57f-0165-437e-9c43-059fb1890e2b" path="/var/lib/kubelet/pods/0183e57f-0165-437e-9c43-059fb1890e2b/volumes"
Oct 30 13:26:54.905416 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:54.905334    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"]
Oct 30 13:26:54.910166 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:26:54.910138    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e2d089d_9c32_44be_ac77_4e9f415e46b0.slice/crio-e18d3efe70a6fb04d072d3794fb9e2a3cd7defb14f3a7883c3f85bbba2f46f7b WatchSource:0}: Error finding container e18d3efe70a6fb04d072d3794fb9e2a3cd7defb14f3a7883c3f85bbba2f46f7b: Status 404 returned error can't find the container with id e18d3efe70a6fb04d072d3794fb9e2a3cd7defb14f3a7883c3f85bbba2f46f7b
Oct 30 13:26:55.092311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:55.092269    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s" event={"ID":"8e2d089d-9c32-44be-ac77-4e9f415e46b0","Type":"ContainerStarted","Data":"e18d3efe70a6fb04d072d3794fb9e2a3cd7defb14f3a7883c3f85bbba2f46f7b"}
Oct 30 13:26:56.098917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:56.098881    5781 generic.go:356] "Generic (PLEG): container finished" podID="8e2d089d-9c32-44be-ac77-4e9f415e46b0" containerID="3a1c83b7c8808daf722f822aba69ad481ea515c1aa3bc68de5882330febede73" exitCode=0
Oct 30 13:26:56.099307 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:56.098958    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s" event={"ID":"8e2d089d-9c32-44be-ac77-4e9f415e46b0","Type":"ContainerDied","Data":"3a1c83b7c8808daf722f822aba69ad481ea515c1aa3bc68de5882330febede73"}
Oct 30 13:26:57.234871 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.234849    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:57.380206 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.380132    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e2d089d-9c32-44be-ac77-4e9f415e46b0-kube-api-access-vmvp4\" (UniqueName: \"kubernetes.io/projected/8e2d089d-9c32-44be-ac77-4e9f415e46b0-kube-api-access-vmvp4\") pod \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") "
Oct 30 13:26:57.380206 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.380186    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-xks-ca\") pod \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") "
Oct 30 13:26:57.380450 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.380218    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-openshift-ca\") pod \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\" (UID: \"8e2d089d-9c32-44be-ac77-4e9f415e46b0\") "
Oct 30 13:26:57.380641 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.380616    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-openshift-ca" pod "8e2d089d-9c32-44be-ac77-4e9f415e46b0" (UID: "8e2d089d-9c32-44be-ac77-4e9f415e46b0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:26:57.382094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.382069    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e2d089d-9c32-44be-ac77-4e9f415e46b0-kube-api-access-vmvp4" pod "8e2d089d-9c32-44be-ac77-4e9f415e46b0" (UID: "8e2d089d-9c32-44be-ac77-4e9f415e46b0"). InnerVolumeSpecName "kube-api-access-vmvp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:26:57.382190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.382072    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-xks-ca" pod "8e2d089d-9c32-44be-ac77-4e9f415e46b0" (UID: "8e2d089d-9c32-44be-ac77-4e9f415e46b0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:26:57.481014 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.480980    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmvp4\" (UniqueName: \"kubernetes.io/projected/8e2d089d-9c32-44be-ac77-4e9f415e46b0-kube-api-access-vmvp4\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:57.481014 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.481011    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:57.481199 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:57.481023    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e2d089d-9c32-44be-ac77-4e9f415e46b0-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:26:58.110494 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:58.110465    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"
Oct 30 13:26:58.110677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:58.110498    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s" event={"ID":"8e2d089d-9c32-44be-ac77-4e9f415e46b0","Type":"ContainerDied","Data":"e18d3efe70a6fb04d072d3794fb9e2a3cd7defb14f3a7883c3f85bbba2f46f7b"}
Oct 30 13:26:58.110677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:26:58.110552    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e18d3efe70a6fb04d072d3794fb9e2a3cd7defb14f3a7883c3f85bbba2f46f7b"
Oct 30 13:27:01.993907 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:27:01.993871    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:27:04.265200 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:04.265167    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:21.828812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.828774    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"]
Oct 30 13:27:21.840233 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.840202    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-zhv7s"]
Oct 30 13:27:21.888548 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.888515    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"]
Oct 30 13:27:21.889012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.888989    5781 cpu_manager.go:414] "containerMap: removing container" podUID="8e2d089d-9c32-44be-ac77-4e9f415e46b0" containerName="revoke-keys"
Oct 30 13:27:21.889012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.889010    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e2d089d-9c32-44be-ac77-4e9f415e46b0" containerName="revoke-keys"
Oct 30 13:27:21.889204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.889118    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e2d089d-9c32-44be-ac77-4e9f415e46b0" containerName="revoke-keys"
Oct 30 13:27:21.892328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.892307    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:21.906927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.906905    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpv7g\" (UniqueName: \"kubernetes.io/projected/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-kube-api-access-wpv7g\") pod \"maas-api-revoke-sub-3d4156f1-gf9rh\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:21.907022 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.906989    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-gf9rh\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:21.907198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.907180    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-gf9rh\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:21.917699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.917661    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"]
Oct 30 13:27:21.917991 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.917969    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:27:21.918302 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:21.918274    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:27:22.008052 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:22.008015    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpv7g\" (UniqueName: \"kubernetes.io/projected/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-kube-api-access-wpv7g\") pod \"maas-api-revoke-sub-3d4156f1-gf9rh\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:22.008179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:22.008081    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-gf9rh\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:22.008179 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:22.008109    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-gf9rh\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:22.008730 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:22.008709    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-gf9rh\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:22.010229 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:22.010209    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-gf9rh\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:22.040703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:22.040684    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpv7g\" (UniqueName: \"kubernetes.io/projected/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-kube-api-access-wpv7g\") pod \"maas-api-revoke-sub-3d4156f1-gf9rh\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:22.204258 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:22.204241    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:22.507196 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:22.506988    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"]
Oct 30 13:27:22.510908 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:27:22.510882    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf234fdb1_e94d_4e6e_a0ad_b46d404a245a.slice/crio-5f38c798c3d393fd107c12b2654f6c49f0fa4b79a176e06e652466aa2756e223 WatchSource:0}: Error finding container 5f38c798c3d393fd107c12b2654f6c49f0fa4b79a176e06e652466aa2756e223: Status 404 returned error can't find the container with id 5f38c798c3d393fd107c12b2654f6c49f0fa4b79a176e06e652466aa2756e223
Oct 30 13:27:22.846495 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:22.846463    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e2d089d-9c32-44be-ac77-4e9f415e46b0" path="/var/lib/kubelet/pods/8e2d089d-9c32-44be-ac77-4e9f415e46b0/volumes"
Oct 30 13:27:23.236175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:23.236138    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh" event={"ID":"f234fdb1-e94d-4e6e-a0ad-b46d404a245a","Type":"ContainerStarted","Data":"bf9b0f2f2bfd0348c69ef3a864b18799022bfb88cb4f2df659a758954c158356"}
Oct 30 13:27:23.236376 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:23.236195    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh" event={"ID":"f234fdb1-e94d-4e6e-a0ad-b46d404a245a","Type":"ContainerStarted","Data":"5f38c798c3d393fd107c12b2654f6c49f0fa4b79a176e06e652466aa2756e223"}
Oct 30 13:27:23.276950 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:23.276903    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh" podStartSLOduration=2.2768899129999998 podStartE2EDuration="2.276889913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:27:23.276128613 +0000 UTC m=+1780.573346156" watchObservedRunningTime="2026-10-30 13:27:23.276889913 +0000 UTC m=+1780.574107475"
Oct 30 13:27:24.243216 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:24.243177    5781 generic.go:356] "Generic (PLEG): container finished" podID="f234fdb1-e94d-4e6e-a0ad-b46d404a245a" containerID="bf9b0f2f2bfd0348c69ef3a864b18799022bfb88cb4f2df659a758954c158356" exitCode=0
Oct 30 13:27:24.243656 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:24.243256    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh" event={"ID":"f234fdb1-e94d-4e6e-a0ad-b46d404a245a","Type":"ContainerDied","Data":"bf9b0f2f2bfd0348c69ef3a864b18799022bfb88cb4f2df659a758954c158356"}
Oct 30 13:27:25.384639 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.384617    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:25.438789 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.438755    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-kube-api-access-wpv7g\" (UniqueName: \"kubernetes.io/projected/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-kube-api-access-wpv7g\") pod \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") "
Oct 30 13:27:25.438954 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.438864    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-openshift-ca\") pod \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") "
Oct 30 13:27:25.439036 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.438952    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-xks-ca\") pod \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\" (UID: \"f234fdb1-e94d-4e6e-a0ad-b46d404a245a\") "
Oct 30 13:27:25.439212 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.439186    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-openshift-ca" pod "f234fdb1-e94d-4e6e-a0ad-b46d404a245a" (UID: "f234fdb1-e94d-4e6e-a0ad-b46d404a245a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:27:25.439355 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.439336    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:27:25.440940 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.440915    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-xks-ca" pod "f234fdb1-e94d-4e6e-a0ad-b46d404a245a" (UID: "f234fdb1-e94d-4e6e-a0ad-b46d404a245a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:27:25.440940 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.440925    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-kube-api-access-wpv7g" pod "f234fdb1-e94d-4e6e-a0ad-b46d404a245a" (UID: "f234fdb1-e94d-4e6e-a0ad-b46d404a245a"). InnerVolumeSpecName "kube-api-access-wpv7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:27:25.540073 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.539999    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpv7g\" (UniqueName: \"kubernetes.io/projected/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-kube-api-access-wpv7g\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:27:25.540073 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:25.540024    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f234fdb1-e94d-4e6e-a0ad-b46d404a245a-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:27:26.256935 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:26.256904    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"
Oct 30 13:27:26.257094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:26.256904    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh" event={"ID":"f234fdb1-e94d-4e6e-a0ad-b46d404a245a","Type":"ContainerDied","Data":"5f38c798c3d393fd107c12b2654f6c49f0fa4b79a176e06e652466aa2756e223"}
Oct 30 13:27:26.257094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:26.257025    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f38c798c3d393fd107c12b2654f6c49f0fa4b79a176e06e652466aa2756e223"
Oct 30 13:27:31.994277 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:27:31.994241    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:27:32.707310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:32.707260    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:37.297676 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:37.297640    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:43.073692 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.073664    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:27:43.075271 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.075246    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:27:43.524639 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.524601    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:43.940619 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.940584    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"]
Oct 30 13:27:43.941400 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.941372    5781 cpu_manager.go:414] "containerMap: removing container" podUID="f234fdb1-e94d-4e6e-a0ad-b46d404a245a" containerName="revoke-keys"
Oct 30 13:27:43.941400 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.941394    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f234fdb1-e94d-4e6e-a0ad-b46d404a245a" containerName="revoke-keys"
Oct 30 13:27:43.941543 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.941462    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="f234fdb1-e94d-4e6e-a0ad-b46d404a245a" containerName="revoke-keys"
Oct 30 13:27:43.945048 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.945029    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:43.959979 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.959956    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-3eceab-openshift-default-dockercfg-nw8lh\""
Oct 30 13:27:43.973640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:43.973601    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"]
Oct 30 13:27:44.001698 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.001638    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:44.006341 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.006318    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-envoy\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.006512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.006354    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e9d552-a398-482e-96f1-0a046068d7db-istiod-ca-cert\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.006512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.006377    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pvtj\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-kube-api-access-5pvtj\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.006587 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.006522    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-credential-socket\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.006587 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.006548    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e9d552-a398-482e-96f1-0a046068d7db-istio-podinfo\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.006587 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.006573    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-istio-token\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.006680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.006630    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-certs\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.006680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.006676    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-socket\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.006742 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.006703    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-data\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.107641 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.107604    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-credential-socket\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.107641 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.107644    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e9d552-a398-482e-96f1-0a046068d7db-istio-podinfo\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.107662    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-istio-token\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.107785    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-certs\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.107879    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-socket\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.107929    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-data\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.107992    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-envoy\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.108040    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e9d552-a398-482e-96f1-0a046068d7db-istiod-ca-cert\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.108057    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-credential-socket\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.108081    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pvtj\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-kube-api-access-5pvtj\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108136 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.108117    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-certs\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108622 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.108178    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-socket\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108622 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.108313    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-data\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.108689 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.108640    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e9d552-a398-482e-96f1-0a046068d7db-istiod-ca-cert\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.110161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.110138    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e9d552-a398-482e-96f1-0a046068d7db-istio-podinfo\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.110161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.110140    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-envoy\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.129997 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.129974    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-istio-token\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.130436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.130416    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pvtj\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-kube-api-access-5pvtj\") pod \"e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") " pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.255270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.255175    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:44.257463 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.257442    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:44.574513 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.574331    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"]
Oct 30 13:27:44.578232 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:27:44.578204    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e9d552_a398_482e_96f1_0a046068d7db.slice/crio-a0fd6a67075121308e8e9774c608c44c238ec9fc06b21c0693c0df9d2bad3395 WatchSource:0}: Error finding container a0fd6a67075121308e8e9774c608c44c238ec9fc06b21c0693c0df9d2bad3395: Status 404 returned error can't find the container with id a0fd6a67075121308e8e9774c608c44c238ec9fc06b21c0693c0df9d2bad3395
Oct 30 13:27:44.580336 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.580306    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:27:44.580435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.580369    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:27:44.580435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:44.580412    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:27:45.146156 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:45.146127    5781 scope.go:122] "RemoveContainer" containerID="89967fec7c56ee7c284f88de383da1f3db325333c830d0be45c15f14832e0647"
Oct 30 13:27:45.154567 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:45.154549    5781 scope.go:122] "RemoveContainer" containerID="ab2f4b44ad86cc64daf008b9739b4f14458aac02913b966040aeaba491f81e61"
Oct 30 13:27:45.164078 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:45.164043    5781 scope.go:122] "RemoveContainer" containerID="62be05d1045c41d36dc9ff81f16e6ebb0e74ea2cd5a946b1398483332e091aaf"
Oct 30 13:27:45.363351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:45.363321    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" event={"ID":"c6e9d552-a398-482e-96f1-0a046068d7db","Type":"ContainerStarted","Data":"9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a"}
Oct 30 13:27:45.363499 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:45.363371    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" event={"ID":"c6e9d552-a398-482e-96f1-0a046068d7db","Type":"ContainerStarted","Data":"a0fd6a67075121308e8e9774c608c44c238ec9fc06b21c0693c0df9d2bad3395"}
Oct 30 13:27:45.430938 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:45.430900    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" podStartSLOduration=2.430888281 podStartE2EDuration="2.430888281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:27:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:27:45.418581073 +0000 UTC m=+1802.715798615" watchObservedRunningTime="2026-10-30 13:27:45.430888281 +0000 UTC m=+1802.728105877"
Oct 30 13:27:46.258270 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:46.258222    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:46.263407 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:46.263380    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:46.368348 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:46.368318    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:46.368986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:46.368964    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:27:58.007810 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:58.007733    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:58.186552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:58.186516    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:59.682997 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:59.682964    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:59.839142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:59.839113    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:59.991097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:59.991031    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:27:59.995599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:27:59.995555    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"]
Oct 30 13:28:00.001033 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.001017    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:00.025024 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.025004    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"]
Oct 30 13:28:00.029255 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.029228    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-3eceab\""
Oct 30 13:28:00.078986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.078939    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"]
Oct 30 13:28:00.082960 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.082939    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:00.103242 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.103217    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"]
Oct 30 13:28:00.110632 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.110610    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-5804d4fa-dockercfg-62krh\""
Oct 30 13:28:00.158079 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.158048    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a18896-3ecc-4a89-9388-1bd5e813d54b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw\" (UID: \"14a18896-3ecc-4a89-9388-1bd5e813d54b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:00.158190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.158107    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ncjt\" (UniqueName: \"kubernetes.io/projected/14a18896-3ecc-4a89-9388-1bd5e813d54b-kube-api-access-5ncjt\") pod \"payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw\" (UID: \"14a18896-3ecc-4a89-9388-1bd5e813d54b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:00.259541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.259475    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj\" (UID: \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:00.259541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.259509    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a18896-3ecc-4a89-9388-1bd5e813d54b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw\" (UID: \"14a18896-3ecc-4a89-9388-1bd5e813d54b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:00.259541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.259535    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ncjt\" (UniqueName: \"kubernetes.io/projected/14a18896-3ecc-4a89-9388-1bd5e813d54b-kube-api-access-5ncjt\") pod \"payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw\" (UID: \"14a18896-3ecc-4a89-9388-1bd5e813d54b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:00.259809 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.259697    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpphw\" (UniqueName: \"kubernetes.io/projected/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-kube-api-access-bpphw\") pod \"payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj\" (UID: \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:00.260246 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.260228    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a18896-3ecc-4a89-9388-1bd5e813d54b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw\" (UID: \"14a18896-3ecc-4a89-9388-1bd5e813d54b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:00.284827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.284798    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ncjt\" (UniqueName: \"kubernetes.io/projected/14a18896-3ecc-4a89-9388-1bd5e813d54b-kube-api-access-5ncjt\") pod \"payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw\" (UID: \"14a18896-3ecc-4a89-9388-1bd5e813d54b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:00.314654 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.314636    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:00.360409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.360380    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpphw\" (UniqueName: \"kubernetes.io/projected/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-kube-api-access-bpphw\") pod \"payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj\" (UID: \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:00.360568 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.360460    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj\" (UID: \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:00.361175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.361150    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj\" (UID: \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:00.394301 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.394268    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpphw\" (UniqueName: \"kubernetes.io/projected/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-kube-api-access-bpphw\") pod \"payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj\" (UID: \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:00.402374 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.402351    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:00.617924 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.617854    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"]
Oct 30 13:28:00.621406 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:28:00.621380    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14a18896_3ecc_4a89_9388_1bd5e813d54b.slice/crio-ece25bae92d62ab0992311f6a3be94b415c9dd1766c3e27969e2fdc226e34324 WatchSource:0}: Error finding container ece25bae92d62ab0992311f6a3be94b415c9dd1766c3e27969e2fdc226e34324: Status 404 returned error can't find the container with id ece25bae92d62ab0992311f6a3be94b415c9dd1766c3e27969e2fdc226e34324
Oct 30 13:28:00.715457 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:00.715424    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"]
Oct 30 13:28:00.719247 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:28:00.719223    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a0ccd02_54a9_446d_a2f0_49f5aa79b7fa.slice/crio-3921af2c647c42e0aa0cef4a39fe0176c4798c702ae67da1dd56eba2b1b0024b WatchSource:0}: Error finding container 3921af2c647c42e0aa0cef4a39fe0176c4798c702ae67da1dd56eba2b1b0024b: Status 404 returned error can't find the container with id 3921af2c647c42e0aa0cef4a39fe0176c4798c702ae67da1dd56eba2b1b0024b
Oct 30 13:28:01.436421 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:01.436374    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:01.453740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:01.453702    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj" event={"ID":"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa","Type":"ContainerStarted","Data":"9ecf9326a5ea9424c5a1e36a98f1958425528c0933f01f7720a1f69123f6eaa9"}
Oct 30 13:28:01.453945 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:01.453765    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj" event={"ID":"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa","Type":"ContainerStarted","Data":"3921af2c647c42e0aa0cef4a39fe0176c4798c702ae67da1dd56eba2b1b0024b"}
Oct 30 13:28:01.453945 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:01.453893    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:01.455379 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:01.455357    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw" event={"ID":"14a18896-3ecc-4a89-9388-1bd5e813d54b","Type":"ContainerStarted","Data":"3a88ac66166fbebf5bba8f86afdc5322e3329a6ed0be14e616a75621e031c1c1"}
Oct 30 13:28:01.455497 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:01.455394    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw" event={"ID":"14a18896-3ecc-4a89-9388-1bd5e813d54b","Type":"ContainerStarted","Data":"ece25bae92d62ab0992311f6a3be94b415c9dd1766c3e27969e2fdc226e34324"}
Oct 30 13:28:01.455497 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:01.455437    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:01.501490 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:01.501444    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj" podStartSLOduration=1.501433649 podStartE2EDuration="1.501433649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:28:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:28:01.499083069 +0000 UTC m=+1818.796300621" watchObservedRunningTime="2026-10-30 13:28:01.501433649 +0000 UTC m=+1818.798651185"
Oct 30 13:28:01.538947 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:01.538911    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw" podStartSLOduration=2.538900901 podStartE2EDuration="2.538900901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:27:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:28:01.537670958 +0000 UTC m=+1818.834888504" watchObservedRunningTime="2026-10-30 13:28:01.538900901 +0000 UTC m=+1818.836118435"
Oct 30 13:28:01.995450 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:28:01.995413    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:28:03.058687 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:03.058634    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:04.851333 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:04.851273    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:06.441783 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:06.441744    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:08.111415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:08.111384    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:09.922981 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:09.922941    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:11.528756 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:11.528717    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:11.974548 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:11.974521    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:12.465394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:12.465366    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:28:12.465620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:12.465432    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:28:12.522181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:12.522102    5781 generic.go:356] "Generic (PLEG): container finished" podID="c6e9d552-a398-482e-96f1-0a046068d7db" containerID="9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a" exitCode=0
Oct 30 13:28:12.522379 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:12.522171    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" event={"ID":"c6e9d552-a398-482e-96f1-0a046068d7db","Type":"ContainerDied","Data":"9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a"}
Oct 30 13:28:12.522607 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:12.522591    5781 scope.go:122] "RemoveContainer" containerID="9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a"
Oct 30 13:28:12.524031 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:12.523997    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:28:12.524127 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:12.524084    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:28:12.524175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:12.524131    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:28:13.077813 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:13.077778    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:13.543030 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:13.542996    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" event={"ID":"c6e9d552-a398-482e-96f1-0a046068d7db","Type":"ContainerStarted","Data":"98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057"}
Oct 30 13:28:14.258330 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:14.258272    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:28:14.259442 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:14.259411    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:28:14.259572 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:14.259463    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 13:28:14.865187 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:14.865152    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:15.258385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:15.258349    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:28:15.258891 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:15.258401    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 13:28:16.258574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:16.258537    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:28:16.258974 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:16.258590    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 13:28:16.337356 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:16.337322    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:16.368894 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:16.368868    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:28:17.258537 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:17.258503    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:28:17.258708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:17.258555    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 13:28:17.877182 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:17.877150    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:18.258129 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:18.258089    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 13:28:18.258347 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:18.258146    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 13:28:19.261828 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:19.261799    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:28:19.262923 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:19.262898    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:28:19.534504 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:19.534421    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:21.167173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:21.167136    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:22.851722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:22.851690    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:23.021340 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:23.021309    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:23.766346 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:23.766317    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:23.957814 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:23.957781    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:24.673078 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:24.673046    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:25.705493 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:25.705457    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:26.234453 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:26.234409    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:26.725432 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:26.725406    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:28.105523 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:28.105489    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:29.641599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:29.641569    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:31.339989 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:31.339947    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:31.995832 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:28:31.995793    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:28:33.037459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:33.037423    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:34.738225 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:34.738198    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:36.271233 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:36.271198    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:37.441781 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:37.441743    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:37.857640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:37.857556    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:39.647336 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:39.647279    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:41.242112 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:41.242056    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:43.020703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:43.020663    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:44.637387 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:44.637349    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:46.364994 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:46.364955    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:48.056837 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:48.056795    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:48.921223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:48.921189    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"]
Oct 30 13:28:48.925393 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:48.925377    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:48.941942 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:48.941925    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:28:48.953648 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:48.953631    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:28:48.955745 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:48.955724    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"]
Oct 30 13:28:48.976712 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:48.976678    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p28zm\" (UniqueName: \"kubernetes.io/projected/d862aae4-cacf-4843-8667-32c83ac41501-kube-api-access-p28zm\") pod \"maas-api-revoke-sub-58a1b7a1-x4zzh\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:48.976827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:48.976790    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-58a1b7a1-x4zzh\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:48.976896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:48.976851    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-xks-ca\") pod \"maas-api-revoke-sub-58a1b7a1-x4zzh\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:49.077571 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.077546    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p28zm\" (UniqueName: \"kubernetes.io/projected/d862aae4-cacf-4843-8667-32c83ac41501-kube-api-access-p28zm\") pod \"maas-api-revoke-sub-58a1b7a1-x4zzh\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:49.077971 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.077593    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-58a1b7a1-x4zzh\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:49.077971 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.077612    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-xks-ca\") pod \"maas-api-revoke-sub-58a1b7a1-x4zzh\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:49.078317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.078264    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-58a1b7a1-x4zzh\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:49.079827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.079811    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-xks-ca\") pod \"maas-api-revoke-sub-58a1b7a1-x4zzh\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:49.100366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.100342    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p28zm\" (UniqueName: \"kubernetes.io/projected/d862aae4-cacf-4843-8667-32c83ac41501-kube-api-access-p28zm\") pod \"maas-api-revoke-sub-58a1b7a1-x4zzh\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:49.238930 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.238880    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:49.537119 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.537037    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"]
Oct 30 13:28:49.540936 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:28:49.540903    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd862aae4_cacf_4843_8667_32c83ac41501.slice/crio-97113973f2984d300490240f2e35c3a8de93f1e9d0abef2a1a4d0e85690ee75f WatchSource:0}: Error finding container 97113973f2984d300490240f2e35c3a8de93f1e9d0abef2a1a4d0e85690ee75f: Status 404 returned error can't find the container with id 97113973f2984d300490240f2e35c3a8de93f1e9d0abef2a1a4d0e85690ee75f
Oct 30 13:28:49.550688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.550665    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:49.733447 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:49.733418    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh" event={"ID":"d862aae4-cacf-4843-8667-32c83ac41501","Type":"ContainerStarted","Data":"97113973f2984d300490240f2e35c3a8de93f1e9d0abef2a1a4d0e85690ee75f"}
Oct 30 13:28:50.741063 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:50.741035    5781 generic.go:356] "Generic (PLEG): container finished" podID="d862aae4-cacf-4843-8667-32c83ac41501" containerID="40cc57907f86a4d62463aec9a7cacee803d3fbb684586806a017d3737bf858d0" exitCode=0
Oct 30 13:28:50.741490 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:50.741086    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh" event={"ID":"d862aae4-cacf-4843-8667-32c83ac41501","Type":"ContainerDied","Data":"40cc57907f86a4d62463aec9a7cacee803d3fbb684586806a017d3737bf858d0"}
Oct 30 13:28:51.245460 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:51.245420    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:51.877588 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:51.877565    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:52.004040 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.003970    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-xks-ca\") pod \"d862aae4-cacf-4843-8667-32c83ac41501\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") "
Oct 30 13:28:52.004040 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.004008    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-openshift-ca\") pod \"d862aae4-cacf-4843-8667-32c83ac41501\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") "
Oct 30 13:28:52.004241 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.004044    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d862aae4-cacf-4843-8667-32c83ac41501-kube-api-access-p28zm\" (UniqueName: \"kubernetes.io/projected/d862aae4-cacf-4843-8667-32c83ac41501-kube-api-access-p28zm\") pod \"d862aae4-cacf-4843-8667-32c83ac41501\" (UID: \"d862aae4-cacf-4843-8667-32c83ac41501\") "
Oct 30 13:28:52.004362 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.004342    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-openshift-ca" pod "d862aae4-cacf-4843-8667-32c83ac41501" (UID: "d862aae4-cacf-4843-8667-32c83ac41501"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:28:52.006107 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.006084    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d862aae4-cacf-4843-8667-32c83ac41501-kube-api-access-p28zm" pod "d862aae4-cacf-4843-8667-32c83ac41501" (UID: "d862aae4-cacf-4843-8667-32c83ac41501"). InnerVolumeSpecName "kube-api-access-p28zm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:28:52.006240 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.006167    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-xks-ca" pod "d862aae4-cacf-4843-8667-32c83ac41501" (UID: "d862aae4-cacf-4843-8667-32c83ac41501"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:28:52.104798 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.104778    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:28:52.104798 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.104798    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d862aae4-cacf-4843-8667-32c83ac41501-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:28:52.104919 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.104807    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p28zm\" (UniqueName: \"kubernetes.io/projected/d862aae4-cacf-4843-8667-32c83ac41501-kube-api-access-p28zm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:28:52.752532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.752499    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh" event={"ID":"d862aae4-cacf-4843-8667-32c83ac41501","Type":"ContainerDied","Data":"97113973f2984d300490240f2e35c3a8de93f1e9d0abef2a1a4d0e85690ee75f"}
Oct 30 13:28:52.752532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.752532    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"
Oct 30 13:28:52.752751 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.752583    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97113973f2984d300490240f2e35c3a8de93f1e9d0abef2a1a4d0e85690ee75f"
Oct 30 13:28:52.797208 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:52.797160    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:54.403986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:54.403958    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:55.926461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:55.926420    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:57.524077 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:57.524046    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:59.172598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:59.172563    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:28:59.989899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:28:59.989866    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:00.127756 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.127722    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"]
Oct 30 13:29:00.128351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.128302    5781 cpu_manager.go:414] "containerMap: removing container" podUID="d862aae4-cacf-4843-8667-32c83ac41501" containerName="revoke-keys"
Oct 30 13:29:00.128351 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.128324    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d862aae4-cacf-4843-8667-32c83ac41501" containerName="revoke-keys"
Oct 30 13:29:00.128530 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.128477    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="d862aae4-cacf-4843-8667-32c83ac41501" containerName="revoke-keys"
Oct 30 13:29:00.131898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.131878    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.153823 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.153793    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"]
Oct 30 13:29:00.202380 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.202347    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:29:00.202380 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.202355    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:29:00.275673 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.275604    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.275673 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.275637    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.275673 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.275658    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv6qn\" (UniqueName: \"kubernetes.io/projected/8a39aacd-666a-4d65-9212-f3ce870178f5-kube-api-access-mv6qn\") pod \"maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.377106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.377068    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.377106 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.377106    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.377368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.377129    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv6qn\" (UniqueName: \"kubernetes.io/projected/8a39aacd-666a-4d65-9212-f3ce870178f5-kube-api-access-mv6qn\") pod \"maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.377762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.377737    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.379367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.379346    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.404723 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.404691    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv6qn\" (UniqueName: \"kubernetes.io/projected/8a39aacd-666a-4d65-9212-f3ce870178f5-kube-api-access-mv6qn\") pod \"maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.445462 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.445423    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:00.743327 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.743252    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"]
Oct 30 13:29:00.748432 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:29:00.748397    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a39aacd_666a_4d65_9212_f3ce870178f5.slice/crio-3094ddfc5728d7afaaa6f917f7db7ee5133d22ed3985eef5015708af513b2c5e WatchSource:0}: Error finding container 3094ddfc5728d7afaaa6f917f7db7ee5133d22ed3985eef5015708af513b2c5e: Status 404 returned error can't find the container with id 3094ddfc5728d7afaaa6f917f7db7ee5133d22ed3985eef5015708af513b2c5e
Oct 30 13:29:00.795322 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.795281    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf" event={"ID":"8a39aacd-666a-4d65-9212-f3ce870178f5","Type":"ContainerStarted","Data":"3094ddfc5728d7afaaa6f917f7db7ee5133d22ed3985eef5015708af513b2c5e"}
Oct 30 13:29:00.855459 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:00.855411    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:01.801920 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:01.801842    5781 generic.go:356] "Generic (PLEG): container finished" podID="8a39aacd-666a-4d65-9212-f3ce870178f5" containerID="53415c147af3a4672d2e44bcb0496cd4206248a039796c119baa26af362e2fc6" exitCode=0
Oct 30 13:29:01.802308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:01.801922    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf" event={"ID":"8a39aacd-666a-4d65-9212-f3ce870178f5","Type":"ContainerDied","Data":"53415c147af3a4672d2e44bcb0496cd4206248a039796c119baa26af362e2fc6"}
Oct 30 13:29:01.994232 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:29:01.994199    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:29:02.510271 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:02.510240    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:02.944509 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:02.944484    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:03.013577 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.013545    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-openshift-ca\") pod \"8a39aacd-666a-4d65-9212-f3ce870178f5\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") "
Oct 30 13:29:03.013739 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.013672    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a39aacd-666a-4d65-9212-f3ce870178f5-kube-api-access-mv6qn\" (UniqueName: \"kubernetes.io/projected/8a39aacd-666a-4d65-9212-f3ce870178f5-kube-api-access-mv6qn\") pod \"8a39aacd-666a-4d65-9212-f3ce870178f5\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") "
Oct 30 13:29:03.013820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.013804    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-xks-ca\") pod \"8a39aacd-666a-4d65-9212-f3ce870178f5\" (UID: \"8a39aacd-666a-4d65-9212-f3ce870178f5\") "
Oct 30 13:29:03.013907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.013884    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-openshift-ca" pod "8a39aacd-666a-4d65-9212-f3ce870178f5" (UID: "8a39aacd-666a-4d65-9212-f3ce870178f5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:29:03.014174 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.014155    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:03.015867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.015840    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a39aacd-666a-4d65-9212-f3ce870178f5-kube-api-access-mv6qn" pod "8a39aacd-666a-4d65-9212-f3ce870178f5" (UID: "8a39aacd-666a-4d65-9212-f3ce870178f5"). InnerVolumeSpecName "kube-api-access-mv6qn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:29:03.015867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.015857    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-xks-ca" pod "8a39aacd-666a-4d65-9212-f3ce870178f5" (UID: "8a39aacd-666a-4d65-9212-f3ce870178f5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:29:03.115173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.115115    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8a39aacd-666a-4d65-9212-f3ce870178f5-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:03.115173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.115138    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv6qn\" (UniqueName: \"kubernetes.io/projected/8a39aacd-666a-4d65-9212-f3ce870178f5-kube-api-access-mv6qn\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:03.814333 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.814272    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf" event={"ID":"8a39aacd-666a-4d65-9212-f3ce870178f5","Type":"ContainerDied","Data":"3094ddfc5728d7afaaa6f917f7db7ee5133d22ed3985eef5015708af513b2c5e"}
Oct 30 13:29:03.814487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.814353    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3094ddfc5728d7afaaa6f917f7db7ee5133d22ed3985eef5015708af513b2c5e"
Oct 30 13:29:03.814487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:03.814311    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"
Oct 30 13:29:04.218666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:04.218630    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:05.027395 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:05.027362    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:05.813010 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:05.812976    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:07.487748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:07.487698    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:09.106876 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:09.106836    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:10.575020 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:10.574983    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"]
Oct 30 13:29:10.607788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:10.607755    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"]
Oct 30 13:29:10.710140 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:10.710105    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:10.871412 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:10.871336    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:12.094916 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:12.094884    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:12.376338 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:12.376241    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:12.716145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:12.716112    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:12.803007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:12.802981    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"]
Oct 30 13:29:12.817433 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:12.817404    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-3eceab-ncnhf"]
Oct 30 13:29:12.840079 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:12.840045    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a39aacd-666a-4d65-9212-f3ce870178f5" path="/var/lib/kubelet/pods/8a39aacd-666a-4d65-9212-f3ce870178f5/volumes"
Oct 30 13:29:12.887091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:12.887061    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:13.428687 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:13.428656    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:15.575910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:15.575840    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj" podUID="0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" containerName="payload-processing" containerID="cri-o://9ecf9326a5ea9424c5a1e36a98f1958425528c0933f01f7720a1f69123f6eaa9" gracePeriod=25
Oct 30 13:29:15.608846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:15.608800    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw" podUID="14a18896-3ecc-4a89-9388-1bd5e813d54b" containerName="payload-pre-processing" containerID="cri-o://3a88ac66166fbebf5bba8f86afdc5322e3329a6ed0be14e616a75621e031c1c1" gracePeriod=25
Oct 30 13:29:20.679543 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:20.679510    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:20.835618 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:20.835578    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:21.039651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:21.039549    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"]
Oct 30 13:29:21.039941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:21.039833    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy" containerID="cri-o://98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057" gracePeriod=30
Oct 30 13:29:21.108666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:21.108635    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:22.369349 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:22.369258    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"]
Oct 30 13:29:22.380237 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:22.380209    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-gf9rh"]
Oct 30 13:29:22.463471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:22.463436    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw" podUID="14a18896-3ecc-4a89-9388-1bd5e813d54b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:29:22.463471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:22.463450    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj" podUID="0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:29:22.840383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:22.840349    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f234fdb1-e94d-4e6e-a0ad-b46d404a245a" path="/var/lib/kubelet/pods/f234fdb1-e94d-4e6e-a0ad-b46d404a245a/volumes"
Oct 30 13:29:26.290800 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.290774    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:29:26.342450 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342382    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-socket\") pod \"c6e9d552-a398-482e-96f1-0a046068d7db\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") "
Oct 30 13:29:26.342450 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342428    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-istio-token\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-istio-token\") pod \"c6e9d552-a398-482e-96f1-0a046068d7db\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") "
Oct 30 13:29:26.342628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342466    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-kube-api-access-5pvtj\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-kube-api-access-5pvtj\") pod \"c6e9d552-a398-482e-96f1-0a046068d7db\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") "
Oct 30 13:29:26.342628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342522    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c6e9d552-a398-482e-96f1-0a046068d7db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e9d552-a398-482e-96f1-0a046068d7db-istio-podinfo\") pod \"c6e9d552-a398-482e-96f1-0a046068d7db\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") "
Oct 30 13:29:26.342733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342628    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-credential-socket\") pod \"c6e9d552-a398-482e-96f1-0a046068d7db\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") "
Oct 30 13:29:26.342733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342639    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-socket" pod "c6e9d552-a398-482e-96f1-0a046068d7db" (UID: "c6e9d552-a398-482e-96f1-0a046068d7db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:29:26.342733 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342716    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6e9d552-a398-482e-96f1-0a046068d7db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e9d552-a398-482e-96f1-0a046068d7db-istiod-ca-cert\") pod \"c6e9d552-a398-482e-96f1-0a046068d7db\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") "
Oct 30 13:29:26.342890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342796    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-envoy\") pod \"c6e9d552-a398-482e-96f1-0a046068d7db\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") "
Oct 30 13:29:26.342890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342849    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-data\") pod \"c6e9d552-a398-482e-96f1-0a046068d7db\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") "
Oct 30 13:29:26.342890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342859    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-credential-socket" pod "c6e9d552-a398-482e-96f1-0a046068d7db" (UID: "c6e9d552-a398-482e-96f1-0a046068d7db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:29:26.342890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.342879    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-certs\") pod \"c6e9d552-a398-482e-96f1-0a046068d7db\" (UID: \"c6e9d552-a398-482e-96f1-0a046068d7db\") "
Oct 30 13:29:26.343092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.343067    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-data" pod "c6e9d552-a398-482e-96f1-0a046068d7db" (UID: "c6e9d552-a398-482e-96f1-0a046068d7db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:29:26.343143 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.343082    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6e9d552-a398-482e-96f1-0a046068d7db-istiod-ca-cert" pod "c6e9d552-a398-482e-96f1-0a046068d7db" (UID: "c6e9d552-a398-482e-96f1-0a046068d7db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:29:26.343268 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.343248    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-certs" pod "c6e9d552-a398-482e-96f1-0a046068d7db" (UID: "c6e9d552-a398-482e-96f1-0a046068d7db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:29:26.343363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.343268    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.343363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.343302    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c6e9d552-a398-482e-96f1-0a046068d7db-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.343363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.343316    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.343363 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.343328    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.344712 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.344669    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c6e9d552-a398-482e-96f1-0a046068d7db-istio-podinfo" pod "c6e9d552-a398-482e-96f1-0a046068d7db" (UID: "c6e9d552-a398-482e-96f1-0a046068d7db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:29:26.344846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.344770    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-kube-api-access-5pvtj" pod "c6e9d552-a398-482e-96f1-0a046068d7db" (UID: "c6e9d552-a398-482e-96f1-0a046068d7db"). InnerVolumeSpecName "kube-api-access-5pvtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:29:26.345042 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.345018    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-istio-token" pod "c6e9d552-a398-482e-96f1-0a046068d7db" (UID: "c6e9d552-a398-482e-96f1-0a046068d7db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:29:26.345241 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.345221    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-envoy" pod "c6e9d552-a398-482e-96f1-0a046068d7db" (UID: "c6e9d552-a398-482e-96f1-0a046068d7db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:29:26.444860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.444817    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c6e9d552-a398-482e-96f1-0a046068d7db-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.444860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.444844    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.444860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.444853    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c6e9d552-a398-482e-96f1-0a046068d7db-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.444860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.444860    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.444860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.444868    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pvtj\" (UniqueName: \"kubernetes.io/projected/c6e9d552-a398-482e-96f1-0a046068d7db-kube-api-access-5pvtj\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:26.933923 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.933895    5781 generic.go:356] "Generic (PLEG): container finished" podID="c6e9d552-a398-482e-96f1-0a046068d7db" containerID="98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057" exitCode=0
Oct 30 13:29:26.934058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.933941    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"
Oct 30 13:29:26.934058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.933975    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" event={"ID":"c6e9d552-a398-482e-96f1-0a046068d7db","Type":"ContainerDied","Data":"98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057"}
Oct 30 13:29:26.934058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.934028    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq" event={"ID":"c6e9d552-a398-482e-96f1-0a046068d7db","Type":"ContainerDied","Data":"a0fd6a67075121308e8e9774c608c44c238ec9fc06b21c0693c0df9d2bad3395"}
Oct 30 13:29:26.934058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.934057    5781 scope.go:122] "RemoveContainer" containerID="98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057"
Oct 30 13:29:26.942695 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.942677    5781 scope.go:122] "RemoveContainer" containerID="9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a"
Oct 30 13:29:26.950474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.950455    5781 scope.go:122] "RemoveContainer" containerID="98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057"
Oct 30 13:29:26.950718 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:29:26.950699    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057\": container with ID starting with 98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057 not found: ID does not exist" containerID="98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057"
Oct 30 13:29:26.950765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.950727    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057"} err="failed to get container status \"98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057\": rpc error: code = NotFound desc = could not find container \"98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057\": container with ID starting with 98a8786834408384b4e512ea048a428a11dfff523f0c234ef685c440765f1057 not found: ID does not exist"
Oct 30 13:29:26.950765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.950744    5781 scope.go:122] "RemoveContainer" containerID="9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a"
Oct 30 13:29:26.950964 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:29:26.950949    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a\": container with ID starting with 9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a not found: ID does not exist" containerID="9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a"
Oct 30 13:29:26.951004 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:26.950969    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a"} err="failed to get container status \"9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a\": rpc error: code = NotFound desc = could not find container \"9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a\": container with ID starting with 9e888896d7bcb7cce802e10180adc0874a917414d3f498ad8cc18b2211c04d8a not found: ID does not exist"
Oct 30 13:29:27.002385 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:27.002361    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"]
Oct 30 13:29:27.012973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:27.012950    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-3eceab-openshift-default-7588556677c9jrq"]
Oct 30 13:29:28.839486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:28.839454    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" path="/var/lib/kubelet/pods/c6e9d552-a398-482e-96f1-0a046068d7db/volumes"
Oct 30 13:29:31.993700 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:29:31.993666    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:29:32.463212 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:32.463179    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj" podUID="0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:29:32.463398 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:32.463187    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw" podUID="14a18896-3ecc-4a89-9388-1bd5e813d54b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:29:36.019486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.018948    5781 generic.go:356] "Generic (PLEG): container finished" podID="0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" containerID="9ecf9326a5ea9424c5a1e36a98f1958425528c0933f01f7720a1f69123f6eaa9" exitCode=0
Oct 30 13:29:36.019486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.019109    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj" event={"ID":"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa","Type":"ContainerDied","Data":"9ecf9326a5ea9424c5a1e36a98f1958425528c0933f01f7720a1f69123f6eaa9"}
Oct 30 13:29:36.021840 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.021692    5781 generic.go:356] "Generic (PLEG): container finished" podID="14a18896-3ecc-4a89-9388-1bd5e813d54b" containerID="3a88ac66166fbebf5bba8f86afdc5322e3329a6ed0be14e616a75621e031c1c1" exitCode=0
Oct 30 13:29:36.021840 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.021773    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw" event={"ID":"14a18896-3ecc-4a89-9388-1bd5e813d54b","Type":"ContainerDied","Data":"3a88ac66166fbebf5bba8f86afdc5322e3329a6ed0be14e616a75621e031c1c1"}
Oct 30 13:29:36.195720 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.195434    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:29:36.267884 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.267849    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:36.268628 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.268601    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14a18896-3ecc-4a89-9388-1bd5e813d54b-kube-api-access-5ncjt\" (UniqueName: \"kubernetes.io/projected/14a18896-3ecc-4a89-9388-1bd5e813d54b-kube-api-access-5ncjt\") pod \"14a18896-3ecc-4a89-9388-1bd5e813d54b\" (UID: \"14a18896-3ecc-4a89-9388-1bd5e813d54b\") "
Oct 30 13:29:36.288486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.285389    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14a18896-3ecc-4a89-9388-1bd5e813d54b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a18896-3ecc-4a89-9388-1bd5e813d54b-plugins-config-volume\") pod \"14a18896-3ecc-4a89-9388-1bd5e813d54b\" (UID: \"14a18896-3ecc-4a89-9388-1bd5e813d54b\") "
Oct 30 13:29:36.288486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.286199    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14a18896-3ecc-4a89-9388-1bd5e813d54b-plugins-config-volume" pod "14a18896-3ecc-4a89-9388-1bd5e813d54b" (UID: "14a18896-3ecc-4a89-9388-1bd5e813d54b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:29:36.292755 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.292731    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14a18896-3ecc-4a89-9388-1bd5e813d54b-kube-api-access-5ncjt" pod "14a18896-3ecc-4a89-9388-1bd5e813d54b" (UID: "14a18896-3ecc-4a89-9388-1bd5e813d54b"). InnerVolumeSpecName "kube-api-access-5ncjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:29:36.301506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.301484    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:29:36.391780 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.391743    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-kube-api-access-bpphw\" (UniqueName: \"kubernetes.io/projected/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-kube-api-access-bpphw\") pod \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\" (UID: \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\") "
Oct 30 13:29:36.391967 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.391811    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-plugins-config-volume\") pod \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\" (UID: \"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa\") "
Oct 30 13:29:36.392317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.392261    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ncjt\" (UniqueName: \"kubernetes.io/projected/14a18896-3ecc-4a89-9388-1bd5e813d54b-kube-api-access-5ncjt\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:36.392453 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.392318    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14a18896-3ecc-4a89-9388-1bd5e813d54b-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:36.392726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.392700    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-plugins-config-volume" pod "0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" (UID: "0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:29:36.394852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.394817    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-kube-api-access-bpphw" pod "0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" (UID: "0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa"). InnerVolumeSpecName "kube-api-access-bpphw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:29:36.494141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.494022    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpphw\" (UniqueName: \"kubernetes.io/projected/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-kube-api-access-bpphw\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:36.494141 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:36.494075    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:29:37.028611 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.028577    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"
Oct 30 13:29:37.029050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.028616    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj" event={"ID":"0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa","Type":"ContainerDied","Data":"3921af2c647c42e0aa0cef4a39fe0176c4798c702ae67da1dd56eba2b1b0024b"}
Oct 30 13:29:37.029050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.028678    5781 scope.go:122] "RemoveContainer" containerID="9ecf9326a5ea9424c5a1e36a98f1958425528c0933f01f7720a1f69123f6eaa9"
Oct 30 13:29:37.030161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.030135    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw" event={"ID":"14a18896-3ecc-4a89-9388-1bd5e813d54b","Type":"ContainerDied","Data":"ece25bae92d62ab0992311f6a3be94b415c9dd1766c3e27969e2fdc226e34324"}
Oct 30 13:29:37.030267 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.030134    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"
Oct 30 13:29:37.037787 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.037768    5781 scope.go:122] "RemoveContainer" containerID="3a88ac66166fbebf5bba8f86afdc5322e3329a6ed0be14e616a75621e031c1c1"
Oct 30 13:29:37.078532 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.078507    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"]
Oct 30 13:29:37.089429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.089404    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-3eceab-5c65b8d57kl6rj"]
Oct 30 13:29:37.130744 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.130674    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"]
Oct 30 13:29:37.134110 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.134087    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-3eceab-78d4dgcskw"]
Oct 30 13:29:37.843984 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:37.843946    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:38.840222 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:38.840190    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" path="/var/lib/kubelet/pods/0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa/volumes"
Oct 30 13:29:38.840611 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:38.840553    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14a18896-3ecc-4a89-9388-1bd5e813d54b" path="/var/lib/kubelet/pods/14a18896-3ecc-4a89-9388-1bd5e813d54b/volumes"
Oct 30 13:29:41.994703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:41.994667    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:43.055321 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:43.055273    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:29:49.476756 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:29:49.476709    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:30:00.167743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.167708    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"]
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168235    5781 cpu_manager.go:414] "containerMap: removing container" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168251    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168263    5781 cpu_manager.go:414] "containerMap: removing container" podUID="14a18896-3ecc-4a89-9388-1bd5e813d54b" containerName="payload-pre-processing"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168279    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14a18896-3ecc-4a89-9388-1bd5e813d54b" containerName="payload-pre-processing"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168302    5781 cpu_manager.go:414] "containerMap: removing container" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168307    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168313    5781 cpu_manager.go:414] "containerMap: removing container" podUID="8a39aacd-666a-4d65-9212-f3ce870178f5" containerName="revoke-keys"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168318    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a39aacd-666a-4d65-9212-f3ce870178f5" containerName="revoke-keys"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168347    5781 cpu_manager.go:414] "containerMap: removing container" podUID="0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" containerName="payload-processing"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168354    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" containerName="payload-processing"
Oct 30 13:30:00.168429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168424    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a0ccd02-54a9-446d-a2f0-49f5aa79b7fa" containerName="payload-processing"
Oct 30 13:30:00.168778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168439    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy"
Oct 30 13:30:00.168778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168446    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="14a18896-3ecc-4a89-9388-1bd5e813d54b" containerName="payload-pre-processing"
Oct 30 13:30:00.168778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168452    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6e9d552-a398-482e-96f1-0a046068d7db" containerName="istio-proxy"
Oct 30 13:30:00.168778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.168457    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a39aacd-666a-4d65-9212-f3ce870178f5" containerName="revoke-keys"
Oct 30 13:30:00.173359 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.173341    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:00.190264 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.190242    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:30:00.204474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.204446    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"]
Oct 30 13:30:00.324515 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.324484    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlzkp\" (UniqueName: \"kubernetes.io/projected/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-kube-api-access-xlzkp\") pod \"maas-api-key-cleanup-29889450-2rlgx\" (UID: \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:00.324660 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.324611    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889450-2rlgx\" (UID: \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:00.425225 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.425202    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlzkp\" (UniqueName: \"kubernetes.io/projected/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-kube-api-access-xlzkp\") pod \"maas-api-key-cleanup-29889450-2rlgx\" (UID: \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:00.425371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.425265    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889450-2rlgx\" (UID: \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:00.425824 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.425805    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889450-2rlgx\" (UID: \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:00.449174 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.449147    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlzkp\" (UniqueName: \"kubernetes.io/projected/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-kube-api-access-xlzkp\") pod \"maas-api-key-cleanup-29889450-2rlgx\" (UID: \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:00.485670 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.485653    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:00.783955 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:00.783889    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"]
Oct 30 13:30:00.788557 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:30:00.788529    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a42f5e6_7b68_47ca_89d9_2311fa6d030f.slice/crio-021d67dd5e7188325335bd6f41120fa1c791cc5c70d9fb9fc2c168305e0b0d12 WatchSource:0}: Error finding container 021d67dd5e7188325335bd6f41120fa1c791cc5c70d9fb9fc2c168305e0b0d12: Status 404 returned error can't find the container with id 021d67dd5e7188325335bd6f41120fa1c791cc5c70d9fb9fc2c168305e0b0d12
Oct 30 13:30:01.156722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:01.156694    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" event={"ID":"5a42f5e6-7b68-47ca-89d9-2311fa6d030f","Type":"ContainerStarted","Data":"6b0219e150d57b7981a0c00677cb38676fcbe56b16fbf2e7190c68bf18ef1519"}
Oct 30 13:30:01.156874 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:01.156740    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" event={"ID":"5a42f5e6-7b68-47ca-89d9-2311fa6d030f","Type":"ContainerStarted","Data":"021d67dd5e7188325335bd6f41120fa1c791cc5c70d9fb9fc2c168305e0b0d12"}
Oct 30 13:30:01.198026 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:01.197984    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" podStartSLOduration=1.197972403 podStartE2EDuration="1.197972403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:30:01.196789065 +0000 UTC m=+1938.494006608" watchObservedRunningTime="2026-10-30 13:30:01.197972403 +0000 UTC m=+1938.495189982"
Oct 30 13:30:01.994887 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:30:01.994848    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:30:02.163403 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:02.163367    5781 generic.go:356] "Generic (PLEG): container finished" podID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerID="6b0219e150d57b7981a0c00677cb38676fcbe56b16fbf2e7190c68bf18ef1519" exitCode=60
Oct 30 13:30:02.163549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:02.163443    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" event={"ID":"5a42f5e6-7b68-47ca-89d9-2311fa6d030f","Type":"ContainerDied","Data":"6b0219e150d57b7981a0c00677cb38676fcbe56b16fbf2e7190c68bf18ef1519"}
Oct 30 13:30:02.163675 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:02.163661    5781 scope.go:122] "RemoveContainer" containerID="6b0219e150d57b7981a0c00677cb38676fcbe56b16fbf2e7190c68bf18ef1519"
Oct 30 13:30:03.170549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:03.170516    5781 generic.go:356] "Generic (PLEG): container finished" podID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerID="44ecf8720de5bc11d22e2b7198c1f9b89f645ccb8da06f1f6790694bcb1aa6de" exitCode=60
Oct 30 13:30:03.171005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:03.170558    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" event={"ID":"5a42f5e6-7b68-47ca-89d9-2311fa6d030f","Type":"ContainerDied","Data":"44ecf8720de5bc11d22e2b7198c1f9b89f645ccb8da06f1f6790694bcb1aa6de"}
Oct 30 13:30:03.171005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:03.170602    5781 scope.go:122] "RemoveContainer" containerID="6b0219e150d57b7981a0c00677cb38676fcbe56b16fbf2e7190c68bf18ef1519"
Oct 30 13:30:03.171005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:03.170826    5781 scope.go:122] "RemoveContainer" containerID="44ecf8720de5bc11d22e2b7198c1f9b89f645ccb8da06f1f6790694bcb1aa6de"
Oct 30 13:30:03.171169 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:30:03.171051    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889450-2rlgx_odh-ai-gateway-infra(5a42f5e6-7b68-47ca-89d9-2311fa6d030f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f"
Oct 30 13:30:04.177142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:04.177111    5781 scope.go:122] "RemoveContainer" containerID="44ecf8720de5bc11d22e2b7198c1f9b89f645ccb8da06f1f6790694bcb1aa6de"
Oct 30 13:30:04.177551 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:30:04.177410    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889450-2rlgx_odh-ai-gateway-infra(5a42f5e6-7b68-47ca-89d9-2311fa6d030f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f"
Oct 30 13:30:12.213705 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:12.213671    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:30:13.834196 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:13.834169    5781 scope.go:122] "RemoveContainer" containerID="44ecf8720de5bc11d22e2b7198c1f9b89f645ccb8da06f1f6790694bcb1aa6de"
Oct 30 13:30:14.229472 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:14.229439    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" event={"ID":"5a42f5e6-7b68-47ca-89d9-2311fa6d030f","Type":"ContainerStarted","Data":"a454183a997d616d40d23a6b94ef4edc6b2246ff17fb5ffe1c7611b208211056"}
Oct 30 13:30:15.236584 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:15.236552    5781 generic.go:356] "Generic (PLEG): container finished" podID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerID="a454183a997d616d40d23a6b94ef4edc6b2246ff17fb5ffe1c7611b208211056" exitCode=60
Oct 30 13:30:15.236998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:15.236623    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" event={"ID":"5a42f5e6-7b68-47ca-89d9-2311fa6d030f","Type":"ContainerDied","Data":"a454183a997d616d40d23a6b94ef4edc6b2246ff17fb5ffe1c7611b208211056"}
Oct 30 13:30:15.236998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:15.236680    5781 scope.go:122] "RemoveContainer" containerID="44ecf8720de5bc11d22e2b7198c1f9b89f645ccb8da06f1f6790694bcb1aa6de"
Oct 30 13:30:15.236998 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:15.236967    5781 scope.go:122] "RemoveContainer" containerID="a454183a997d616d40d23a6b94ef4edc6b2246ff17fb5ffe1c7611b208211056"
Oct 30 13:30:15.237272 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:30:15.237255    5781 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889450-2rlgx_odh-ai-gateway-infra(5a42f5e6-7b68-47ca-89d9-2311fa6d030f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f"
Oct 30 13:30:15.290102 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:15.290070    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"]
Oct 30 13:30:16.376492 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:16.376466    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:16.479604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:16.479577    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-kube-api-access-xlzkp\" (UniqueName: \"kubernetes.io/projected/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-kube-api-access-xlzkp\") pod \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\" (UID: \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\") "
Oct 30 13:30:16.479760 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:16.479656    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-maas-api-service-ca\") pod \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\" (UID: \"5a42f5e6-7b68-47ca-89d9-2311fa6d030f\") "
Oct 30 13:30:16.480074 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:16.480047    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-maas-api-service-ca" pod "5a42f5e6-7b68-47ca-89d9-2311fa6d030f" (UID: "5a42f5e6-7b68-47ca-89d9-2311fa6d030f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:30:16.481533 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:16.481509    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-kube-api-access-xlzkp" pod "5a42f5e6-7b68-47ca-89d9-2311fa6d030f" (UID: "5a42f5e6-7b68-47ca-89d9-2311fa6d030f"). InnerVolumeSpecName "kube-api-access-xlzkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:30:16.581031 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:16.580973    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-maas-api-service-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:30:16.581031 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:16.581000    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlzkp\" (UniqueName: \"kubernetes.io/projected/5a42f5e6-7b68-47ca-89d9-2311fa6d030f-kube-api-access-xlzkp\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:30:17.249314 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:17.249272    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"
Oct 30 13:30:17.249473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:17.249316    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx" event={"ID":"5a42f5e6-7b68-47ca-89d9-2311fa6d030f","Type":"ContainerDied","Data":"021d67dd5e7188325335bd6f41120fa1c791cc5c70d9fb9fc2c168305e0b0d12"}
Oct 30 13:30:17.249473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:17.249374    5781 scope.go:122] "RemoveContainer" containerID="a454183a997d616d40d23a6b94ef4edc6b2246ff17fb5ffe1c7611b208211056"
Oct 30 13:30:17.309692 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:17.309669    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"]
Oct 30 13:30:17.318971 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:17.318949    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889450-2rlgx"]
Oct 30 13:30:18.841044 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:18.841001    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" path="/var/lib/kubelet/pods/5a42f5e6-7b68-47ca-89d9-2311fa6d030f/volumes"
Oct 30 13:30:31.995129 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:30:31.995094    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:30:36.282702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:36.282660    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:30:37.965684 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:37.965650    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:30:47.035931 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:47.035896    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"]
Oct 30 13:30:47.047067 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:47.047033    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a4aef7d7-fbq8w"]
Oct 30 13:30:47.753350 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:47.753316    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:30:48.840386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:30:48.840345    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cea85225-e1cb-465e-9b58-30a1c66576b5" path="/var/lib/kubelet/pods/cea85225-e1cb-465e-9b58-30a1c66576b5/volumes"
Oct 30 13:31:01.996005 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:31:01.995924    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:31:08.034995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:08.034962    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"]
Oct 30 13:31:08.046455 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:08.046425    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-79805ca9-bg29r"]
Oct 30 13:31:08.839890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:08.839857    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32b7861f-eabe-4b9f-be4d-d77194df97e9" path="/var/lib/kubelet/pods/32b7861f-eabe-4b9f-be4d-d77194df97e9/volumes"
Oct 30 13:31:12.382133 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:12.382099    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:31:22.654509 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.654473    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"]
Oct 30 13:31:22.655082 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.655057    5781 cpu_manager.go:414] "containerMap: removing container" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerName="cleanup"
Oct 30 13:31:22.655211 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.655191    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerName="cleanup"
Oct 30 13:31:22.655281 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.655221    5781 cpu_manager.go:414] "containerMap: removing container" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerName="cleanup"
Oct 30 13:31:22.655281 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.655230    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerName="cleanup"
Oct 30 13:31:22.655434 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.655419    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerName="cleanup"
Oct 30 13:31:22.655488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.655437    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerName="cleanup"
Oct 30 13:31:22.655488 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.655460    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerName="cleanup"
Oct 30 13:31:22.658674 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.658654    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.684839 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.684735    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:31:22.685129 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.685105    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:31:22.686630 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.686605    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"]
Oct 30 13:31:22.700344 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.700321    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-vfrzs\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.700466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.700374    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-vfrzs\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.700531 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.700494    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9knpb\" (UniqueName: \"kubernetes.io/projected/f21c1f99-19b0-47b1-b35b-e98e704f0788-kube-api-access-9knpb\") pod \"maas-api-revoke-sub-d576e68b-vfrzs\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.801235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.801200    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-vfrzs\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.801420 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.801252    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-vfrzs\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.801420 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.801284    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9knpb\" (UniqueName: \"kubernetes.io/projected/f21c1f99-19b0-47b1-b35b-e98e704f0788-kube-api-access-9knpb\") pod \"maas-api-revoke-sub-d576e68b-vfrzs\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.801866 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.801845    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d576e68b-vfrzs\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.803534 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.803512    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d576e68b-vfrzs\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.824102 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.824079    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9knpb\" (UniqueName: \"kubernetes.io/projected/f21c1f99-19b0-47b1-b35b-e98e704f0788-kube-api-access-9knpb\") pod \"maas-api-revoke-sub-d576e68b-vfrzs\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:22.971180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:22.971150    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:23.273734 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:23.273658    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"]
Oct 30 13:31:23.278507 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:31:23.278479    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf21c1f99_19b0_47b1_b35b_e98e704f0788.slice/crio-4fd615c12e1e61777ef5515b8897f7aea19e038697d43feae71dc4566b20c0dd WatchSource:0}: Error finding container 4fd615c12e1e61777ef5515b8897f7aea19e038697d43feae71dc4566b20c0dd: Status 404 returned error can't find the container with id 4fd615c12e1e61777ef5515b8897f7aea19e038697d43feae71dc4566b20c0dd
Oct 30 13:31:23.602712 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:23.602681    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs" event={"ID":"f21c1f99-19b0-47b1-b35b-e98e704f0788","Type":"ContainerStarted","Data":"4fd615c12e1e61777ef5515b8897f7aea19e038697d43feae71dc4566b20c0dd"}
Oct 30 13:31:24.608906 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:24.608870    5781 generic.go:356] "Generic (PLEG): container finished" podID="f21c1f99-19b0-47b1-b35b-e98e704f0788" containerID="32988163dfb30472b532267da9679f5eb9d9a9d9b261e48a1749fc8d5efd1b81" exitCode=0
Oct 30 13:31:24.609326 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:24.608949    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs" event={"ID":"f21c1f99-19b0-47b1-b35b-e98e704f0788","Type":"ContainerDied","Data":"32988163dfb30472b532267da9679f5eb9d9a9d9b261e48a1749fc8d5efd1b81"}
Oct 30 13:31:25.034535 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.034403    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"]
Oct 30 13:31:25.037724 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.037701    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b7e8edb-pjh58"]
Oct 30 13:31:25.746122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.746097    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:25.830620 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.830589    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f21c1f99-19b0-47b1-b35b-e98e704f0788-kube-api-access-9knpb\" (UniqueName: \"kubernetes.io/projected/f21c1f99-19b0-47b1-b35b-e98e704f0788-kube-api-access-9knpb\") pod \"f21c1f99-19b0-47b1-b35b-e98e704f0788\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") "
Oct 30 13:31:25.830804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.830682    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-xks-ca\") pod \"f21c1f99-19b0-47b1-b35b-e98e704f0788\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") "
Oct 30 13:31:25.830804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.830710    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-openshift-ca\") pod \"f21c1f99-19b0-47b1-b35b-e98e704f0788\" (UID: \"f21c1f99-19b0-47b1-b35b-e98e704f0788\") "
Oct 30 13:31:25.831122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.831095    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-openshift-ca" pod "f21c1f99-19b0-47b1-b35b-e98e704f0788" (UID: "f21c1f99-19b0-47b1-b35b-e98e704f0788"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:31:25.832655 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.832632    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-xks-ca" pod "f21c1f99-19b0-47b1-b35b-e98e704f0788" (UID: "f21c1f99-19b0-47b1-b35b-e98e704f0788"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:31:25.832754 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.832734    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f21c1f99-19b0-47b1-b35b-e98e704f0788-kube-api-access-9knpb" pod "f21c1f99-19b0-47b1-b35b-e98e704f0788" (UID: "f21c1f99-19b0-47b1-b35b-e98e704f0788"). InnerVolumeSpecName "kube-api-access-9knpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:31:25.931992 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.931957    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:31:25.931992 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.931986    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f21c1f99-19b0-47b1-b35b-e98e704f0788-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:31:25.931992 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:25.931994    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9knpb\" (UniqueName: \"kubernetes.io/projected/f21c1f99-19b0-47b1-b35b-e98e704f0788-kube-api-access-9knpb\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:31:26.620661 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:26.620630    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"
Oct 30 13:31:26.620839 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:26.620629    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs" event={"ID":"f21c1f99-19b0-47b1-b35b-e98e704f0788","Type":"ContainerDied","Data":"4fd615c12e1e61777ef5515b8897f7aea19e038697d43feae71dc4566b20c0dd"}
Oct 30 13:31:26.620839 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:26.620747    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fd615c12e1e61777ef5515b8897f7aea19e038697d43feae71dc4566b20c0dd"
Oct 30 13:31:26.839461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:26.839431    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37001bb8-a1e3-4a93-a2dc-7817a10099d4" path="/var/lib/kubelet/pods/37001bb8-a1e3-4a93-a2dc-7817a10099d4/volumes"
Oct 30 13:31:31.995226 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:31:31.995189    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:31:38.231744 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:38.231708    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:31:39.678072 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.678037    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"]
Oct 30 13:31:39.678604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.678538    5781 cpu_manager.go:414] "containerMap: removing container" podUID="f21c1f99-19b0-47b1-b35b-e98e704f0788" containerName="revoke-keys"
Oct 30 13:31:39.678604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.678552    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f21c1f99-19b0-47b1-b35b-e98e704f0788" containerName="revoke-keys"
Oct 30 13:31:39.678604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.678590    5781 cpu_manager.go:414] "containerMap: removing container" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerName="cleanup"
Oct 30 13:31:39.678604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.678596    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a42f5e6-7b68-47ca-89d9-2311fa6d030f" containerName="cleanup"
Oct 30 13:31:39.678756 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.678667    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="f21c1f99-19b0-47b1-b35b-e98e704f0788" containerName="revoke-keys"
Oct 30 13:31:39.682081 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.682059    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.696409 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.696375    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:31:39.707917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.707876    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:31:39.710266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.710241    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"]
Oct 30 13:31:39.794866 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.794835    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-kpbfm\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.795012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.794938    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26gxh\" (UniqueName: \"kubernetes.io/projected/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-kube-api-access-26gxh\") pod \"maas-api-revoke-sub-d2bc93bc-kpbfm\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.795012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.794970    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-kpbfm\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.896418 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.896386    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-kpbfm\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.896571 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.896479    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26gxh\" (UniqueName: \"kubernetes.io/projected/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-kube-api-access-26gxh\") pod \"maas-api-revoke-sub-d2bc93bc-kpbfm\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.896571 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.896514    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-kpbfm\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.897153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.897133    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d2bc93bc-kpbfm\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.898662 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.898646    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d2bc93bc-kpbfm\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.920792 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.920759    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26gxh\" (UniqueName: \"kubernetes.io/projected/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-kube-api-access-26gxh\") pod \"maas-api-revoke-sub-d2bc93bc-kpbfm\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:39.995194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:39.995133    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:40.291691 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:40.291617    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"]
Oct 30 13:31:40.296103 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:31:40.296076    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcd7986f_13b4_4f5e_9829_d55b29c6f33e.slice/crio-0ac9a0c86fdb0e1dae46dd2af5ea058e520f2e2128618d9d550340107a7363d9 WatchSource:0}: Error finding container 0ac9a0c86fdb0e1dae46dd2af5ea058e520f2e2128618d9d550340107a7363d9: Status 404 returned error can't find the container with id 0ac9a0c86fdb0e1dae46dd2af5ea058e520f2e2128618d9d550340107a7363d9
Oct 30 13:31:40.709844 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:40.709810    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm" event={"ID":"dcd7986f-13b4-4f5e-9829-d55b29c6f33e","Type":"ContainerStarted","Data":"0a29ad06f094e2db7a8d4f51bc9a5db64bce4a4adc2c9471cf0c957cdf3236f1"}
Oct 30 13:31:40.710223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:40.709870    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm" event={"ID":"dcd7986f-13b4-4f5e-9829-d55b29c6f33e","Type":"ContainerStarted","Data":"0ac9a0c86fdb0e1dae46dd2af5ea058e520f2e2128618d9d550340107a7363d9"}
Oct 30 13:31:40.751915 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:40.751873    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm" podStartSLOduration=1.7518587239999999 podStartE2EDuration="1.751858724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:31:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:31:40.750605624 +0000 UTC m=+2038.047823166" watchObservedRunningTime="2026-10-30 13:31:40.751858724 +0000 UTC m=+2038.049076272"
Oct 30 13:31:41.715847 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:41.715814    5781 generic.go:356] "Generic (PLEG): container finished" podID="dcd7986f-13b4-4f5e-9829-d55b29c6f33e" containerID="0a29ad06f094e2db7a8d4f51bc9a5db64bce4a4adc2c9471cf0c957cdf3236f1" exitCode=0
Oct 30 13:31:41.716217 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:41.715896    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm" event={"ID":"dcd7986f-13b4-4f5e-9829-d55b29c6f33e","Type":"ContainerDied","Data":"0a29ad06f094e2db7a8d4f51bc9a5db64bce4a4adc2c9471cf0c957cdf3236f1"}
Oct 30 13:31:42.858466 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:42.858445    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:43.026025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.025963    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-openshift-ca\") pod \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") "
Oct 30 13:31:43.026025 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.026012    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-kube-api-access-26gxh\" (UniqueName: \"kubernetes.io/projected/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-kube-api-access-26gxh\") pod \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") "
Oct 30 13:31:43.026205 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.026098    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-xks-ca\") pod \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\" (UID: \"dcd7986f-13b4-4f5e-9829-d55b29c6f33e\") "
Oct 30 13:31:43.026383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.026352    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-openshift-ca" pod "dcd7986f-13b4-4f5e-9829-d55b29c6f33e" (UID: "dcd7986f-13b4-4f5e-9829-d55b29c6f33e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:31:43.027983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.027962    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-xks-ca" pod "dcd7986f-13b4-4f5e-9829-d55b29c6f33e" (UID: "dcd7986f-13b4-4f5e-9829-d55b29c6f33e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:31:43.028068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.028044    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-kube-api-access-26gxh" pod "dcd7986f-13b4-4f5e-9829-d55b29c6f33e" (UID: "dcd7986f-13b4-4f5e-9829-d55b29c6f33e"). InnerVolumeSpecName "kube-api-access-26gxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:31:43.127556 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.127531    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:31:43.127556 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.127552    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:31:43.127684 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.127561    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26gxh\" (UniqueName: \"kubernetes.io/projected/dcd7986f-13b4-4f5e-9829-d55b29c6f33e-kube-api-access-26gxh\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:31:43.727870 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.727834    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm" event={"ID":"dcd7986f-13b4-4f5e-9829-d55b29c6f33e","Type":"ContainerDied","Data":"0ac9a0c86fdb0e1dae46dd2af5ea058e520f2e2128618d9d550340107a7363d9"}
Oct 30 13:31:43.728047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.727899    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ac9a0c86fdb0e1dae46dd2af5ea058e520f2e2128618d9d550340107a7363d9"
Oct 30 13:31:43.728047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:43.727847    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"
Oct 30 13:31:45.234077 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:45.234042    5781 scope.go:122] "RemoveContainer" containerID="eb4901a608ad25a1d0e8366b370b1eb5e78952cc43bb301ce13b6c1922036192"
Oct 30 13:31:47.820893 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:47.820860    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:31:51.595995 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:51.595961    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:31:52.106589 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.106559    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"]
Oct 30 13:31:52.107155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.107131    5781 cpu_manager.go:414] "containerMap: removing container" podUID="dcd7986f-13b4-4f5e-9829-d55b29c6f33e" containerName="revoke-keys"
Oct 30 13:31:52.107350 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.107335    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcd7986f-13b4-4f5e-9829-d55b29c6f33e" containerName="revoke-keys"
Oct 30 13:31:52.107688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.107673    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcd7986f-13b4-4f5e-9829-d55b29c6f33e" containerName="revoke-keys"
Oct 30 13:31:52.112645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.112626    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.127184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.127163    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-c78854-openshift-default-dockercfg-hwx8n\""
Oct 30 13:31:52.141110 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.141085    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"]
Oct 30 13:31:52.209178 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.209141    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55a482c-2288-4209-b9ec-a0539f0958ec-istiod-ca-cert\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.209362 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.209195    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xffbn\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-kube-api-access-xffbn\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.209362 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.209271    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-podinfo\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.209442 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.209367    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-socket\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.209527 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.209439    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-certs\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.209527 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.209488    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-credential-socket\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.209608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.209545    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-token\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.209608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.209585    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-envoy\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.209700 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.209627    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-data\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.210273 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.210244    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:31:52.310099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310072    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-token\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310249 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310104    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-envoy\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310249 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310142    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-data\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310249 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310174    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55a482c-2288-4209-b9ec-a0539f0958ec-istiod-ca-cert\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310249 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310212    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xffbn\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-kube-api-access-xffbn\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310249 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310236    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-podinfo\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310593 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310262    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-socket\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310593 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310353    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-certs\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310593 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310407    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-credential-socket\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310613    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-data\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310723    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-certs\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310809    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-credential-socket\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.310953 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310931    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-socket\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.311013 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.310965    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55a482c-2288-4209-b9ec-a0539f0958ec-istiod-ca-cert\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.312439 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.312421    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-envoy\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.312788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.312766    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-podinfo\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.340384 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.340354    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-token\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.341669 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.341648    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xffbn\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-kube-api-access-xffbn\") pod \"e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") " pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.378530 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.378468    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:31:52.425585 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.425561    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:52.741224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.741191    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"]
Oct 30 13:31:52.745155 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:31:52.745131    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd55a482c_2288_4209_b9ec_a0539f0958ec.slice/crio-4b3e640eed81dbeece4ae79c79a8e8779b46bdcfc6928c37d0365c12114ce35c WatchSource:0}: Error finding container 4b3e640eed81dbeece4ae79c79a8e8779b46bdcfc6928c37d0365c12114ce35c: Status 404 returned error can't find the container with id 4b3e640eed81dbeece4ae79c79a8e8779b46bdcfc6928c37d0365c12114ce35c
Oct 30 13:31:52.747200 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.747169    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:31:52.747257 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.747236    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:31:52.747311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.747270    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:31:52.773716 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:52.773692    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" event={"ID":"d55a482c-2288-4209-b9ec-a0539f0958ec","Type":"ContainerStarted","Data":"4b3e640eed81dbeece4ae79c79a8e8779b46bdcfc6928c37d0365c12114ce35c"}
Oct 30 13:31:53.781062 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:53.781019    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" event={"ID":"d55a482c-2288-4209-b9ec-a0539f0958ec","Type":"ContainerStarted","Data":"8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1"}
Oct 30 13:31:53.838762 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:53.838710    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" podStartSLOduration=1.838694056 podStartE2EDuration="1.838694056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:31:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:31:53.836667078 +0000 UTC m=+2051.133884618" watchObservedRunningTime="2026-10-30 13:31:53.838694056 +0000 UTC m=+2051.135911600"
Oct 30 13:31:54.426452 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:54.426422    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:54.431262 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:54.431241    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:54.786553 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:54.786470    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:54.787357 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:54.787339    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:31:55.934989 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:55.934961    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:31:56.098832 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:31:56.098804    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:01.995280 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:32:01.995247    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:32:06.417681 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:06.417651    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:06.646001 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:06.645962    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:07.996530 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:07.989818    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:08.197071 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.197039    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:08.306816 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.306742    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"]
Oct 30 13:32:08.310605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.310583    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:08.339012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.338985    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-c78854\""
Oct 30 13:32:08.339135 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.339029    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-f68a0166-dockercfg-lrrp7\""
Oct 30 13:32:08.346974 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.346950    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"]
Oct 30 13:32:08.402865 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.402831    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:08.466718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.466688    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td6cg\" (UniqueName: \"kubernetes.io/projected/926dd644-f6fb-4ca8-85d9-c416a157a7c1-kube-api-access-td6cg\") pod \"payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz\" (UID: \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:08.466910 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.466757    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/926dd644-f6fb-4ca8-85d9-c416a157a7c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz\" (UID: \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:08.471915 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.471883    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"]
Oct 30 13:32:08.475665 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.475645    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:08.499584 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.499559    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"]
Oct 30 13:32:08.567812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.567752    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td6cg\" (UniqueName: \"kubernetes.io/projected/926dd644-f6fb-4ca8-85d9-c416a157a7c1-kube-api-access-td6cg\") pod \"payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz\" (UID: \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:08.567812 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.567786    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae783cf-960c-4734-8400-e2e575aa1dba-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq\" (UID: \"2ae783cf-960c-4734-8400-e2e575aa1dba\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:08.568006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.567830    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlsmd\" (UniqueName: \"kubernetes.io/projected/2ae783cf-960c-4734-8400-e2e575aa1dba-kube-api-access-zlsmd\") pod \"payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq\" (UID: \"2ae783cf-960c-4734-8400-e2e575aa1dba\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:08.568006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.567941    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/926dd644-f6fb-4ca8-85d9-c416a157a7c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz\" (UID: \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:08.568593 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.568575    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/926dd644-f6fb-4ca8-85d9-c416a157a7c1-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz\" (UID: \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:08.611824 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.611800    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-td6cg\" (UniqueName: \"kubernetes.io/projected/926dd644-f6fb-4ca8-85d9-c416a157a7c1-kube-api-access-td6cg\") pod \"payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz\" (UID: \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:08.624773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.624753    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:08.669265 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.669229    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae783cf-960c-4734-8400-e2e575aa1dba-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq\" (UID: \"2ae783cf-960c-4734-8400-e2e575aa1dba\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:08.669432 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.669363    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlsmd\" (UniqueName: \"kubernetes.io/projected/2ae783cf-960c-4734-8400-e2e575aa1dba-kube-api-access-zlsmd\") pod \"payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq\" (UID: \"2ae783cf-960c-4734-8400-e2e575aa1dba\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:08.669918 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.669900    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae783cf-960c-4734-8400-e2e575aa1dba-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq\" (UID: \"2ae783cf-960c-4734-8400-e2e575aa1dba\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:08.703916 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.703888    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlsmd\" (UniqueName: \"kubernetes.io/projected/2ae783cf-960c-4734-8400-e2e575aa1dba-kube-api-access-zlsmd\") pod \"payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq\" (UID: \"2ae783cf-960c-4734-8400-e2e575aa1dba\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:08.788638 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.788612    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:08.932122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:08.932091    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"]
Oct 30 13:32:08.935889 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:32:08.935864    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod926dd644_f6fb_4ca8_85d9_c416a157a7c1.slice/crio-42579bc20318e3b8c5d1329bc969aa78357afb042188e29b5fa0a3dfc2a810b1 WatchSource:0}: Error finding container 42579bc20318e3b8c5d1329bc969aa78357afb042188e29b5fa0a3dfc2a810b1: Status 404 returned error can't find the container with id 42579bc20318e3b8c5d1329bc969aa78357afb042188e29b5fa0a3dfc2a810b1
Oct 30 13:32:09.088190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:09.088153    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"]
Oct 30 13:32:09.091954 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:32:09.091906    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae783cf_960c_4734_8400_e2e575aa1dba.slice/crio-70eafba937d715aae5f2df74c8eacfc3863c315b61283ac064419024a600ca51 WatchSource:0}: Error finding container 70eafba937d715aae5f2df74c8eacfc3863c315b61283ac064419024a600ca51: Status 404 returned error can't find the container with id 70eafba937d715aae5f2df74c8eacfc3863c315b61283ac064419024a600ca51
Oct 30 13:32:09.863171 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:09.863136    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq" event={"ID":"2ae783cf-960c-4734-8400-e2e575aa1dba","Type":"ContainerStarted","Data":"737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f"}
Oct 30 13:32:09.863366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:09.863191    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq" event={"ID":"2ae783cf-960c-4734-8400-e2e575aa1dba","Type":"ContainerStarted","Data":"70eafba937d715aae5f2df74c8eacfc3863c315b61283ac064419024a600ca51"}
Oct 30 13:32:09.863366 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:09.863216    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:09.864810 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:09.864768    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz" event={"ID":"926dd644-f6fb-4ca8-85d9-c416a157a7c1","Type":"ContainerStarted","Data":"884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202"}
Oct 30 13:32:09.864928 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:09.864833    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz" event={"ID":"926dd644-f6fb-4ca8-85d9-c416a157a7c1","Type":"ContainerStarted","Data":"42579bc20318e3b8c5d1329bc969aa78357afb042188e29b5fa0a3dfc2a810b1"}
Oct 30 13:32:09.864928 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:09.864864    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:09.906759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:09.906717    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq" podStartSLOduration=1.906703531 podStartE2EDuration="1.906703531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:32:09.904608075 +0000 UTC m=+2067.201825628" watchObservedRunningTime="2026-10-30 13:32:09.906703531 +0000 UTC m=+2067.203921072"
Oct 30 13:32:09.944498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:09.944457    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz" podStartSLOduration=1.9444462470000001 podStartE2EDuration="1.944446247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:32:09.943457142 +0000 UTC m=+2067.240674684" watchObservedRunningTime="2026-10-30 13:32:09.944446247 +0000 UTC m=+2067.241663783"
Oct 30 13:32:10.020573 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:10.020537    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:11.802635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:11.802600    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:13.488857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:13.488824    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:14.995506 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:14.995448    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:16.635312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:16.629217    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:18.858782 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:18.858747    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:20.605552 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:20.605511    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:20.872518 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:20.872444    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:32:20.872518 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:20.872518    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:32:21.450160 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:21.450107    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:21.932099 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:21.932065    5781 generic.go:356] "Generic (PLEG): container finished" podID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerID="8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1" exitCode=0
Oct 30 13:32:21.932508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:21.932132    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" event={"ID":"d55a482c-2288-4209-b9ec-a0539f0958ec","Type":"ContainerDied","Data":"8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1"}
Oct 30 13:32:21.932508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:21.932498    5781 scope.go:122] "RemoveContainer" containerID="8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1"
Oct 30 13:32:21.933536 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:21.933509    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:32:21.933595 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:21.933582    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:32:21.933631 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:21.933612    5781 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:32:22.544599 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:22.544558    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:22.944661 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:22.944630    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" event={"ID":"d55a482c-2288-4209-b9ec-a0539f0958ec","Type":"ContainerStarted","Data":"67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde"}
Oct 30 13:32:23.434378 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:23.434344    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:32:23.435431 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:23.435405    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 13:32:23.435646 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:23.435620    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 13:32:24.040326 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:24.037129    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:24.434371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:24.429650    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 13:32:24.434371 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:24.429708    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 13:32:24.787019 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:24.786943    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:32:25.436313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:25.435379    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 13:32:25.436313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:25.435473    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 13:32:25.538223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:25.538179    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:26.432310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:26.431748    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 13:32:26.432310 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:26.431828    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 13:32:27.238817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:27.238633    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:27.433313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:27.430801    5781 patch_prober.go:29] interesting pod/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 13:32:27.433313 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:27.430869    5781 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 13:32:28.430586 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:28.430554    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:32:28.431666 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:28.431644    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:32:28.874759 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:28.874627    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:30.628096 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:30.628060    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:31.997371 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:32:31.997333    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:32:32.148253 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:32.148214    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:32.329367 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:32.329242    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:33.787481 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:33.787445    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:33.933218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:33.933189    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:35.648512 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:35.648472    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:37.236546 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:37.236511    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:38.334858 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:38.334816    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:39.055335 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:39.055277    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:40.574348 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:40.574304    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:42.325658 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:42.325618    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:43.112176 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:43.112149    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:32:43.115702 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:43.115674    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:32:44.012800 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:44.012763    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:45.256574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:45.256541    5781 scope.go:122] "RemoveContainer" containerID="14e5cce602fb19b9ddda57c7880b80907bc48de6333e1eb42d3b87dc50b0ab7c"
Oct 30 13:32:45.265706 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:45.265684    5781 scope.go:122] "RemoveContainer" containerID="e3853c5c932336cc244abb20c23d04107978c29441a42a6cd05a5e12a0d88a1f"
Oct 30 13:32:45.276490 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:45.276453    5781 scope.go:122] "RemoveContainer" containerID="204c92deda67f6b45f93559fa85bc342f96c8227bd63f83965f60f89658ecda1"
Oct 30 13:32:45.646846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:45.646759    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:47.365148 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:47.365100    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:47.985923 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:47.985888    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:48.994228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:48.994192    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:50.673160 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:50.673126    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:52.206169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:52.206113    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:53.949050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:53.949005    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:55.575843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:55.575804    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:57.084209 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:57.084171    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:32:58.629066 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:32:58.629030    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:00.147181 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:00.147143    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:01.182584 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.182546    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"]
Oct 30 13:33:01.188447 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.188422    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.202740 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.202714    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:33:01.214454 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.214419    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:33:01.216435 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.216400    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"]
Oct 30 13:33:01.262774 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.262718    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btrsc\" (UniqueName: \"kubernetes.io/projected/c2333b40-c133-413f-9146-481564ce441c-kube-api-access-btrsc\") pod \"maas-api-revoke-sub-c4701c16-bdk56\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.262970 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.262854    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2333b40-c133-413f-9146-481564ce441c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c4701c16-bdk56\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.262970 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.262958    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2333b40-c133-413f-9146-481564ce441c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c4701c16-bdk56\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.363926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.363880    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2333b40-c133-413f-9146-481564ce441c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c4701c16-bdk56\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.364448 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.363980    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2333b40-c133-413f-9146-481564ce441c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c4701c16-bdk56\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.365161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.365128    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2333b40-c133-413f-9146-481564ce441c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c4701c16-bdk56\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.365392 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.365358    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btrsc\" (UniqueName: \"kubernetes.io/projected/c2333b40-c133-413f-9146-481564ce441c-kube-api-access-btrsc\") pod \"maas-api-revoke-sub-c4701c16-bdk56\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.367644 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.367612    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2333b40-c133-413f-9146-481564ce441c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c4701c16-bdk56\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.389273 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.389241    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btrsc\" (UniqueName: \"kubernetes.io/projected/c2333b40-c133-413f-9146-481564ce441c-kube-api-access-btrsc\") pod \"maas-api-revoke-sub-c4701c16-bdk56\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.502173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.502085    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:01.811317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.811125    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"]
Oct 30 13:33:01.815653 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:33:01.815623    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2333b40_c133_413f_9146_481564ce441c.slice/crio-6e48243e51163a509892714f0cd292acd76aa023d61bc02796903b08e4c54ffc WatchSource:0}: Error finding container 6e48243e51163a509892714f0cd292acd76aa023d61bc02796903b08e4c54ffc: Status 404 returned error can't find the container with id 6e48243e51163a509892714f0cd292acd76aa023d61bc02796903b08e4c54ffc
Oct 30 13:33:01.842154 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:01.842111    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:01.997395 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:33:01.997355    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:33:02.160245 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:02.160199    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56" event={"ID":"c2333b40-c133-413f-9146-481564ce441c","Type":"ContainerStarted","Data":"e627440de812db0674675fdfb60931a994a8f9828b05e2a9d4ca09f31628b51c"}
Oct 30 13:33:02.160457 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:02.160262    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56" event={"ID":"c2333b40-c133-413f-9146-481564ce441c","Type":"ContainerStarted","Data":"6e48243e51163a509892714f0cd292acd76aa023d61bc02796903b08e4c54ffc"}
Oct 30 13:33:02.200217 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:02.200162    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56" podStartSLOduration=1.200148638 podStartE2EDuration="1.200148638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:33:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:33:02.199438882 +0000 UTC m=+2119.496656437" watchObservedRunningTime="2026-10-30 13:33:02.200148638 +0000 UTC m=+2119.497366182"
Oct 30 13:33:03.168054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:03.167960    5781 generic.go:356] "Generic (PLEG): container finished" podID="c2333b40-c133-413f-9146-481564ce441c" containerID="e627440de812db0674675fdfb60931a994a8f9828b05e2a9d4ca09f31628b51c" exitCode=0
Oct 30 13:33:03.168217 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:03.168056    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56" event={"ID":"c2333b40-c133-413f-9146-481564ce441c","Type":"ContainerDied","Data":"e627440de812db0674675fdfb60931a994a8f9828b05e2a9d4ca09f31628b51c"}
Oct 30 13:33:03.454638 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:03.454599    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:04.323744 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.323715    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:04.397097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.397052    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2333b40-c133-413f-9146-481564ce441c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2333b40-c133-413f-9146-481564ce441c-maas-api-openshift-ca\") pod \"c2333b40-c133-413f-9146-481564ce441c\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") "
Oct 30 13:33:04.397317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.397136    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c2333b40-c133-413f-9146-481564ce441c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2333b40-c133-413f-9146-481564ce441c-maas-api-xks-ca\") pod \"c2333b40-c133-413f-9146-481564ce441c\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") "
Oct 30 13:33:04.397317 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.397187    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2333b40-c133-413f-9146-481564ce441c-kube-api-access-btrsc\" (UniqueName: \"kubernetes.io/projected/c2333b40-c133-413f-9146-481564ce441c-kube-api-access-btrsc\") pod \"c2333b40-c133-413f-9146-481564ce441c\" (UID: \"c2333b40-c133-413f-9146-481564ce441c\") "
Oct 30 13:33:04.397525 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.397501    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2333b40-c133-413f-9146-481564ce441c-maas-api-openshift-ca" pod "c2333b40-c133-413f-9146-481564ce441c" (UID: "c2333b40-c133-413f-9146-481564ce441c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:33:04.397640 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.397620    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/c2333b40-c133-413f-9146-481564ce441c-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:04.399443 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.399415    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2333b40-c133-413f-9146-481564ce441c-maas-api-xks-ca" pod "c2333b40-c133-413f-9146-481564ce441c" (UID: "c2333b40-c133-413f-9146-481564ce441c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:33:04.399878 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.399857    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2333b40-c133-413f-9146-481564ce441c-kube-api-access-btrsc" pod "c2333b40-c133-413f-9146-481564ce441c" (UID: "c2333b40-c133-413f-9146-481564ce441c"). InnerVolumeSpecName "kube-api-access-btrsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:33:04.499256 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.499169    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/c2333b40-c133-413f-9146-481564ce441c-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:04.499256 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:04.499201    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btrsc\" (UniqueName: \"kubernetes.io/projected/c2333b40-c133-413f-9146-481564ce441c-kube-api-access-btrsc\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:05.056907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:05.056868    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:05.179984 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:05.179949    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"
Oct 30 13:33:05.180145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:05.179949    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56" event={"ID":"c2333b40-c133-413f-9146-481564ce441c","Type":"ContainerDied","Data":"6e48243e51163a509892714f0cd292acd76aa023d61bc02796903b08e4c54ffc"}
Oct 30 13:33:05.180145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:05.180076    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e48243e51163a509892714f0cd292acd76aa023d61bc02796903b08e4c54ffc"
Oct 30 13:33:06.560851 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:06.560810    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:08.155941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:08.155903    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:09.761594 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:09.761557    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:11.271685 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:11.271637    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:12.981194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:12.981138    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:14.599654 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:14.599608    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:16.318418 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:16.318377    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:17.480509 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:17.480466    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:17.909264 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:17.909187    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:19.400712 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:19.400678    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:21.220604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:21.220554    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:22.711489 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:22.711459    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:24.319424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:24.319359    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:25.892263 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:25.892225    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:26.095198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.095164    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:26.219767 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.219728    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"]
Oct 30 13:33:26.220408 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.220384    5781 cpu_manager.go:414] "containerMap: removing container" podUID="c2333b40-c133-413f-9146-481564ce441c" containerName="revoke-keys"
Oct 30 13:33:26.220408 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.220406    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2333b40-c133-413f-9146-481564ce441c" containerName="revoke-keys"
Oct 30 13:33:26.220591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.220508    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2333b40-c133-413f-9146-481564ce441c" containerName="revoke-keys"
Oct 30 13:33:26.224159 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.224134    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.251749 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.251714    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:33:26.252191 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.251767    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:33:26.254907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.254880    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"]
Oct 30 13:33:26.314608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.314574    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.314788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.314628    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlgp9\" (UniqueName: \"kubernetes.io/projected/74bf3d80-139d-4d02-af94-7a8e99da70ad-kube-api-access-tlgp9\") pod \"maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.314788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.314761    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.415478 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.415441    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlgp9\" (UniqueName: \"kubernetes.io/projected/74bf3d80-139d-4d02-af94-7a8e99da70ad-kube-api-access-tlgp9\") pod \"maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.415680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.415517    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.415680 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.415582    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.416194 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.416176    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.417869 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.417851    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.438148 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.438123    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlgp9\" (UniqueName: \"kubernetes.io/projected/74bf3d80-139d-4d02-af94-7a8e99da70ad-kube-api-access-tlgp9\") pod \"maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.536977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.536903    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:26.840765 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:33:26.840735    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74bf3d80_139d_4d02_af94_7a8e99da70ad.slice/crio-30c7996b7a54ee13d5402d7cc0080dc7a984b377c0571cabe09c046ee113f5f0 WatchSource:0}: Error finding container 30c7996b7a54ee13d5402d7cc0080dc7a984b377c0571cabe09c046ee113f5f0: Status 404 returned error can't find the container with id 30c7996b7a54ee13d5402d7cc0080dc7a984b377c0571cabe09c046ee113f5f0
Oct 30 13:33:26.840765 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:26.840757    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"]
Oct 30 13:33:27.304847 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:27.297901    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq" event={"ID":"74bf3d80-139d-4d02-af94-7a8e99da70ad","Type":"ContainerStarted","Data":"aae943e584b657bbbc6659626683d66603862184785548912172d8a1c7635e48"}
Oct 30 13:33:27.304847 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:27.298062    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq" event={"ID":"74bf3d80-139d-4d02-af94-7a8e99da70ad","Type":"ContainerStarted","Data":"30c7996b7a54ee13d5402d7cc0080dc7a984b377c0571cabe09c046ee113f5f0"}
Oct 30 13:33:27.341379 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:27.341322    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq" podStartSLOduration=1.341309203 podStartE2EDuration="1.341309203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:33:27.340432074 +0000 UTC m=+2144.637649614" watchObservedRunningTime="2026-10-30 13:33:27.341309203 +0000 UTC m=+2144.638526743"
Oct 30 13:33:27.689901 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:27.689870    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:28.304707 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:28.304673    5781 generic.go:356] "Generic (PLEG): container finished" podID="74bf3d80-139d-4d02-af94-7a8e99da70ad" containerID="aae943e584b657bbbc6659626683d66603862184785548912172d8a1c7635e48" exitCode=0
Oct 30 13:33:28.304883 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:28.304767    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq" event={"ID":"74bf3d80-139d-4d02-af94-7a8e99da70ad","Type":"ContainerDied","Data":"aae943e584b657bbbc6659626683d66603862184785548912172d8a1c7635e48"}
Oct 30 13:33:29.332645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.332600    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:29.453047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.453024    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:29.648541 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.648449    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74bf3d80-139d-4d02-af94-7a8e99da70ad-kube-api-access-tlgp9\" (UniqueName: \"kubernetes.io/projected/74bf3d80-139d-4d02-af94-7a8e99da70ad-kube-api-access-tlgp9\") pod \"74bf3d80-139d-4d02-af94-7a8e99da70ad\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") "
Oct 30 13:33:29.648727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.648539    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-openshift-ca\") pod \"74bf3d80-139d-4d02-af94-7a8e99da70ad\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") "
Oct 30 13:33:29.648727 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.648584    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-xks-ca\") pod \"74bf3d80-139d-4d02-af94-7a8e99da70ad\" (UID: \"74bf3d80-139d-4d02-af94-7a8e99da70ad\") "
Oct 30 13:33:29.648886 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.648859    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-openshift-ca" pod "74bf3d80-139d-4d02-af94-7a8e99da70ad" (UID: "74bf3d80-139d-4d02-af94-7a8e99da70ad"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:33:29.649043 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.649027    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:29.650776 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.650748    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-xks-ca" pod "74bf3d80-139d-4d02-af94-7a8e99da70ad" (UID: "74bf3d80-139d-4d02-af94-7a8e99da70ad"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:33:29.650882 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.650796    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74bf3d80-139d-4d02-af94-7a8e99da70ad-kube-api-access-tlgp9" pod "74bf3d80-139d-4d02-af94-7a8e99da70ad" (UID: "74bf3d80-139d-4d02-af94-7a8e99da70ad"). InnerVolumeSpecName "kube-api-access-tlgp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:33:29.750306 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.750264    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/74bf3d80-139d-4d02-af94-7a8e99da70ad-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:29.750501 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:29.750325    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlgp9\" (UniqueName: \"kubernetes.io/projected/74bf3d80-139d-4d02-af94-7a8e99da70ad-kube-api-access-tlgp9\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:30.316617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:30.316585    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"
Oct 30 13:33:30.316617 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:30.316598    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq" event={"ID":"74bf3d80-139d-4d02-af94-7a8e99da70ad","Type":"ContainerDied","Data":"30c7996b7a54ee13d5402d7cc0080dc7a984b377c0571cabe09c046ee113f5f0"}
Oct 30 13:33:30.316827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:30.316639    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30c7996b7a54ee13d5402d7cc0080dc7a984b377c0571cabe09c046ee113f5f0"
Oct 30 13:33:30.920958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:30.920922    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:31.202744 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:31.202657    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:31.996191 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:33:31.996148    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:33:32.508645 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:32.508610    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:34.052404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:34.052369    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:35.773772 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:35.773738    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:36.694800 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:36.694757    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"]
Oct 30 13:33:36.729473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:36.729449    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"]
Oct 30 13:33:36.883415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:36.883386    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:37.059684 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:37.059611    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:37.245091 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:37.245057    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:38.403961 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:38.403928    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:38.495710 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:38.495676    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"]
Oct 30 13:33:38.507155 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:38.507121    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-single-main-c78854-742vq"]
Oct 30 13:33:38.638639 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:38.638611    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:38.840006 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:38.839980    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74bf3d80-139d-4d02-af94-7a8e99da70ad" path="/var/lib/kubelet/pods/74bf3d80-139d-4d02-af94-7a8e99da70ad/volumes"
Oct 30 13:33:39.177352 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:39.177319    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:41.695058 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:41.695020    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq" podUID="2ae783cf-960c-4734-8400-e2e575aa1dba" containerName="payload-processing" containerID="cri-o://737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f" gracePeriod=25
Oct 30 13:33:41.730616 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:41.730551    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz" podUID="926dd644-f6fb-4ca8-85d9-c416a157a7c1" containerName="payload-pre-processing" containerID="cri-o://884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202" gracePeriod=25
Oct 30 13:33:45.306238 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:45.306208    5781 scope.go:122] "RemoveContainer" containerID="bf9b0f2f2bfd0348c69ef3a864b18799022bfb88cb4f2df659a758954c158356"
Oct 30 13:33:45.314851 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:45.314830    5781 scope.go:122] "RemoveContainer" containerID="3a1c83b7c8808daf722f822aba69ad481ea515c1aa3bc68de5882330febede73"
Oct 30 13:33:46.948361 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:46.948282    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:47.229748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:47.229661    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:47.456975 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:47.456471    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"]
Oct 30 13:33:47.456975 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:47.456678    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy" containerID="cri-o://67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde" gracePeriod=30
Oct 30 13:33:47.855244 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:47.855207    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:48.152542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:48.152465    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:48.558033 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:48.558002    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:33:50.870497 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:50.870457    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz" podUID="926dd644-f6fb-4ca8-85d9-c416a157a7c1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:33:50.870893 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:50.870528    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq" podUID="2ae783cf-960c-4734-8400-e2e575aa1dba" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:33:52.039615 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.039525    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"]
Oct 30 13:33:52.050820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.050793    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-58a1b7a1-x4zzh"]
Oct 30 13:33:52.716951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.716928    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:33:52.777699 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.777633    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-socket\") pod \"d55a482c-2288-4209-b9ec-a0539f0958ec\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") "
Oct 30 13:33:52.777852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.777730    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-credential-socket\") pod \"d55a482c-2288-4209-b9ec-a0539f0958ec\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") "
Oct 30 13:33:52.777852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.777787    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-podinfo\") pod \"d55a482c-2288-4209-b9ec-a0539f0958ec\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") "
Oct 30 13:33:52.777852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.777828    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-envoy\") pod \"d55a482c-2288-4209-b9ec-a0539f0958ec\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") "
Oct 30 13:33:52.777852 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.777852    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d55a482c-2288-4209-b9ec-a0539f0958ec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55a482c-2288-4209-b9ec-a0539f0958ec-istiod-ca-cert\") pod \"d55a482c-2288-4209-b9ec-a0539f0958ec\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") "
Oct 30 13:33:52.778097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.777879    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-token\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-token\") pod \"d55a482c-2288-4209-b9ec-a0539f0958ec\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") "
Oct 30 13:33:52.778097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.777921    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-data\") pod \"d55a482c-2288-4209-b9ec-a0539f0958ec\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") "
Oct 30 13:33:52.778097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.777986    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-certs\") pod \"d55a482c-2288-4209-b9ec-a0539f0958ec\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") "
Oct 30 13:33:52.778097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.778026    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-kube-api-access-xffbn\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-kube-api-access-xffbn\") pod \"d55a482c-2288-4209-b9ec-a0539f0958ec\" (UID: \"d55a482c-2288-4209-b9ec-a0539f0958ec\") "
Oct 30 13:33:52.778097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.778001    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-socket" pod "d55a482c-2288-4209-b9ec-a0539f0958ec" (UID: "d55a482c-2288-4209-b9ec-a0539f0958ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:33:52.778097 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.778087    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-credential-socket" pod "d55a482c-2288-4209-b9ec-a0539f0958ec" (UID: "d55a482c-2288-4209-b9ec-a0539f0958ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:33:52.778471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.778377    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d55a482c-2288-4209-b9ec-a0539f0958ec-istiod-ca-cert" pod "d55a482c-2288-4209-b9ec-a0539f0958ec" (UID: "d55a482c-2288-4209-b9ec-a0539f0958ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:33:52.778471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.778426    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:52.778471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.778447    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:52.778752 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.778723    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-certs" pod "d55a482c-2288-4209-b9ec-a0539f0958ec" (UID: "d55a482c-2288-4209-b9ec-a0539f0958ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:33:52.778976 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.778953    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-data" pod "d55a482c-2288-4209-b9ec-a0539f0958ec" (UID: "d55a482c-2288-4209-b9ec-a0539f0958ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:33:52.780773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.780747    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-token" pod "d55a482c-2288-4209-b9ec-a0539f0958ec" (UID: "d55a482c-2288-4209-b9ec-a0539f0958ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:33:52.781014 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.780991    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-podinfo" pod "d55a482c-2288-4209-b9ec-a0539f0958ec" (UID: "d55a482c-2288-4209-b9ec-a0539f0958ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:33:52.781014 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.780990    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-envoy" pod "d55a482c-2288-4209-b9ec-a0539f0958ec" (UID: "d55a482c-2288-4209-b9ec-a0539f0958ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:33:52.781149 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.781064    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-kube-api-access-xffbn" pod "d55a482c-2288-4209-b9ec-a0539f0958ec" (UID: "d55a482c-2288-4209-b9ec-a0539f0958ec"). InnerVolumeSpecName "kube-api-access-xffbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:33:52.839764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.839735    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d862aae4-cacf-4843-8667-32c83ac41501" path="/var/lib/kubelet/pods/d862aae4-cacf-4843-8667-32c83ac41501/volumes"
Oct 30 13:33:52.879865 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.879840    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:52.879865 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.879862    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:52.880029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.879871    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:52.880029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.879881    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xffbn\" (UniqueName: \"kubernetes.io/projected/d55a482c-2288-4209-b9ec-a0539f0958ec-kube-api-access-xffbn\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:52.880029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.879889    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:52.880029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.879898    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d55a482c-2288-4209-b9ec-a0539f0958ec-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:52.880029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:52.879907    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d55a482c-2288-4209-b9ec-a0539f0958ec-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:33:53.443510 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.443477    5781 generic.go:356] "Generic (PLEG): container finished" podID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerID="67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde" exitCode=0
Oct 30 13:33:53.444016 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.443545    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"
Oct 30 13:33:53.444016 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.443572    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" event={"ID":"d55a482c-2288-4209-b9ec-a0539f0958ec","Type":"ContainerDied","Data":"67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde"}
Oct 30 13:33:53.444016 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.443642    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8" event={"ID":"d55a482c-2288-4209-b9ec-a0539f0958ec","Type":"ContainerDied","Data":"4b3e640eed81dbeece4ae79c79a8e8779b46bdcfc6928c37d0365c12114ce35c"}
Oct 30 13:33:53.444016 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.443678    5781 scope.go:122] "RemoveContainer" containerID="67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde"
Oct 30 13:33:53.452918 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.452891    5781 scope.go:122] "RemoveContainer" containerID="8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1"
Oct 30 13:33:53.461627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.461602    5781 scope.go:122] "RemoveContainer" containerID="67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde"
Oct 30 13:33:53.461906 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:33:53.461877    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde\": container with ID starting with 67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde not found: ID does not exist" containerID="67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde"
Oct 30 13:33:53.461972 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.461915    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde"} err="failed to get container status \"67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde\": rpc error: code = NotFound desc = could not find container \"67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde\": container with ID starting with 67df2213985a6819543382db49ee09e6be397b6b2cee55574ced25fe5aa7bfde not found: ID does not exist"
Oct 30 13:33:53.461972 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.461934    5781 scope.go:122] "RemoveContainer" containerID="8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1"
Oct 30 13:33:53.462166 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:33:53.462146    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1\": container with ID starting with 8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1 not found: ID does not exist" containerID="8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1"
Oct 30 13:33:53.462226 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.462179    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1"} err="failed to get container status \"8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1\": rpc error: code = NotFound desc = could not find container \"8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1\": container with ID starting with 8d5edcad6d1f52300c18d97fe01f4fd41a5f7c362468886466ff5f4e11edafc1 not found: ID does not exist"
Oct 30 13:33:53.511507 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.511476    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"]
Oct 30 13:33:53.525266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:53.525233    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-c78854-openshift-default-66f5b589d5tc8k8"]
Oct 30 13:33:54.840142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:33:54.840109    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" path="/var/lib/kubelet/pods/d55a482c-2288-4209-b9ec-a0539f0958ec/volumes"
Oct 30 13:34:00.870357 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:00.870322    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz" podUID="926dd644-f6fb-4ca8-85d9-c416a157a7c1" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:34:00.870756 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:00.870335    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq" podUID="2ae783cf-960c-4734-8400-e2e575aa1dba" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 13:34:01.682888 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:01.682850    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:02.003499 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:34:02.003461    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:34:02.013206 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.013183    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:34:02.015931 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.015913    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:34:02.169813 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.169785    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/926dd644-f6fb-4ca8-85d9-c416a157a7c1-kube-api-access-td6cg\" (UniqueName: \"kubernetes.io/projected/926dd644-f6fb-4ca8-85d9-c416a157a7c1-kube-api-access-td6cg\") pod \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\" (UID: \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\") "
Oct 30 13:34:02.169813 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.169819    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ae783cf-960c-4734-8400-e2e575aa1dba-kube-api-access-zlsmd\" (UniqueName: \"kubernetes.io/projected/2ae783cf-960c-4734-8400-e2e575aa1dba-kube-api-access-zlsmd\") pod \"2ae783cf-960c-4734-8400-e2e575aa1dba\" (UID: \"2ae783cf-960c-4734-8400-e2e575aa1dba\") "
Oct 30 13:34:02.170090 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.169922    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/926dd644-f6fb-4ca8-85d9-c416a157a7c1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/926dd644-f6fb-4ca8-85d9-c416a157a7c1-plugins-config-volume\") pod \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\" (UID: \"926dd644-f6fb-4ca8-85d9-c416a157a7c1\") "
Oct 30 13:34:02.170090 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.169951    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ae783cf-960c-4734-8400-e2e575aa1dba-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae783cf-960c-4734-8400-e2e575aa1dba-plugins-config-volume\") pod \"2ae783cf-960c-4734-8400-e2e575aa1dba\" (UID: \"2ae783cf-960c-4734-8400-e2e575aa1dba\") "
Oct 30 13:34:02.170410 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.170365    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/926dd644-f6fb-4ca8-85d9-c416a157a7c1-plugins-config-volume" pod "926dd644-f6fb-4ca8-85d9-c416a157a7c1" (UID: "926dd644-f6fb-4ca8-85d9-c416a157a7c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:34:02.170547 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.170436    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ae783cf-960c-4734-8400-e2e575aa1dba-plugins-config-volume" pod "2ae783cf-960c-4734-8400-e2e575aa1dba" (UID: "2ae783cf-960c-4734-8400-e2e575aa1dba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:34:02.172007 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.171984    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ae783cf-960c-4734-8400-e2e575aa1dba-kube-api-access-zlsmd" pod "2ae783cf-960c-4734-8400-e2e575aa1dba" (UID: "2ae783cf-960c-4734-8400-e2e575aa1dba"). InnerVolumeSpecName "kube-api-access-zlsmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:34:02.172094 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.172014    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/926dd644-f6fb-4ca8-85d9-c416a157a7c1-kube-api-access-td6cg" pod "926dd644-f6fb-4ca8-85d9-c416a157a7c1" (UID: "926dd644-f6fb-4ca8-85d9-c416a157a7c1"). InnerVolumeSpecName "kube-api-access-td6cg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:34:02.270846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.270775    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/926dd644-f6fb-4ca8-85d9-c416a157a7c1-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:02.270846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.270804    5781 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ae783cf-960c-4734-8400-e2e575aa1dba-plugins-config-volume\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:02.270846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.270813    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td6cg\" (UniqueName: \"kubernetes.io/projected/926dd644-f6fb-4ca8-85d9-c416a157a7c1-kube-api-access-td6cg\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:02.270846 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.270822    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlsmd\" (UniqueName: \"kubernetes.io/projected/2ae783cf-960c-4734-8400-e2e575aa1dba-kube-api-access-zlsmd\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:02.491934 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.491897    5781 generic.go:356] "Generic (PLEG): container finished" podID="2ae783cf-960c-4734-8400-e2e575aa1dba" containerID="737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f" exitCode=0
Oct 30 13:34:02.492151 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.491949    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"
Oct 30 13:34:02.492151 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.491989    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq" event={"ID":"2ae783cf-960c-4734-8400-e2e575aa1dba","Type":"ContainerDied","Data":"737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f"}
Oct 30 13:34:02.492151 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.492080    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq" event={"ID":"2ae783cf-960c-4734-8400-e2e575aa1dba","Type":"ContainerDied","Data":"70eafba937d715aae5f2df74c8eacfc3863c315b61283ac064419024a600ca51"}
Oct 30 13:34:02.492151 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.492109    5781 scope.go:122] "RemoveContainer" containerID="737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f"
Oct 30 13:34:02.493460 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.493436    5781 generic.go:356] "Generic (PLEG): container finished" podID="926dd644-f6fb-4ca8-85d9-c416a157a7c1" containerID="884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202" exitCode=0
Oct 30 13:34:02.493460 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.493461    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz" event={"ID":"926dd644-f6fb-4ca8-85d9-c416a157a7c1","Type":"ContainerDied","Data":"884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202"}
Oct 30 13:34:02.493635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.493488    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"
Oct 30 13:34:02.493635 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.493504    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz" event={"ID":"926dd644-f6fb-4ca8-85d9-c416a157a7c1","Type":"ContainerDied","Data":"42579bc20318e3b8c5d1329bc969aa78357afb042188e29b5fa0a3dfc2a810b1"}
Oct 30 13:34:02.501239 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.501218    5781 scope.go:122] "RemoveContainer" containerID="737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f"
Oct 30 13:34:02.501580 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:34:02.501560    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f\": container with ID starting with 737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f not found: ID does not exist" containerID="737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f"
Oct 30 13:34:02.501692 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.501591    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f"} err="failed to get container status \"737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f\": rpc error: code = NotFound desc = could not find container \"737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f\": container with ID starting with 737fe8c817fbe9eebf33cd1c0e60c74e21a7717f146403c9f1ccc81484a8d96f not found: ID does not exist"
Oct 30 13:34:02.501692 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.501616    5781 scope.go:122] "RemoveContainer" containerID="884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202"
Oct 30 13:34:02.510029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.510005    5781 scope.go:122] "RemoveContainer" containerID="884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202"
Oct 30 13:34:02.510335 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:34:02.510315    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202\": container with ID starting with 884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202 not found: ID does not exist" containerID="884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202"
Oct 30 13:34:02.510399 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.510345    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202"} err="failed to get container status \"884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202\": rpc error: code = NotFound desc = could not find container \"884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202\": container with ID starting with 884e4ec917c0eafca4c8a7fa8e3e77283f01a93506f04e43bc3d2f869611b202 not found: ID does not exist"
Oct 30 13:34:02.541195 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.541137    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"]
Oct 30 13:34:02.552753 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.552725    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-single-main-c78854-56578k4kwz"]
Oct 30 13:34:02.579388 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.579356    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"]
Oct 30 13:34:02.585954 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.585919    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-c78854-546b55c86tm8dq"]
Oct 30 13:34:02.841557 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.841477    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ae783cf-960c-4734-8400-e2e575aa1dba" path="/var/lib/kubelet/pods/2ae783cf-960c-4734-8400-e2e575aa1dba/volumes"
Oct 30 13:34:02.841856 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:02.841843    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="926dd644-f6fb-4ca8-85d9-c416a157a7c1" path="/var/lib/kubelet/pods/926dd644-f6fb-4ca8-85d9-c416a157a7c1/volumes"
Oct 30 13:34:08.104049 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:08.103970    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:10.011180 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:10.011149    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:16.548387 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:16.548354    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:18.312703 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.312669    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"]
Oct 30 13:34:18.313361 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313342    5781 cpu_manager.go:414] "containerMap: removing container" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy"
Oct 30 13:34:18.313461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313362    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy"
Oct 30 13:34:18.313461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313391    5781 cpu_manager.go:414] "containerMap: removing container" podUID="74bf3d80-139d-4d02-af94-7a8e99da70ad" containerName="revoke-keys"
Oct 30 13:34:18.313461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313401    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74bf3d80-139d-4d02-af94-7a8e99da70ad" containerName="revoke-keys"
Oct 30 13:34:18.313461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313419    5781 cpu_manager.go:414] "containerMap: removing container" podUID="2ae783cf-960c-4734-8400-e2e575aa1dba" containerName="payload-processing"
Oct 30 13:34:18.313461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313427    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ae783cf-960c-4734-8400-e2e575aa1dba" containerName="payload-processing"
Oct 30 13:34:18.313461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313452    5781 cpu_manager.go:414] "containerMap: removing container" podUID="926dd644-f6fb-4ca8-85d9-c416a157a7c1" containerName="payload-pre-processing"
Oct 30 13:34:18.313461 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313460    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="926dd644-f6fb-4ca8-85d9-c416a157a7c1" containerName="payload-pre-processing"
Oct 30 13:34:18.313860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313571    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy"
Oct 30 13:34:18.313860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313587    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="74bf3d80-139d-4d02-af94-7a8e99da70ad" containerName="revoke-keys"
Oct 30 13:34:18.313860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313600    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ae783cf-960c-4734-8400-e2e575aa1dba" containerName="payload-processing"
Oct 30 13:34:18.313860 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.313612    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="926dd644-f6fb-4ca8-85d9-c416a157a7c1" containerName="payload-pre-processing"
Oct 30 13:34:18.316967 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.316948    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.342764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.342740    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-0b6c7bb2-kserve-self-signed-certs\""
Oct 30 13:34:18.344778 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.344755    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"]
Oct 30 13:34:18.376675 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.376649    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-model-cache\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.376788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.376708    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tmp-dir\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.376788 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.376770    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-dshm\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.376865 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.376799    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzzsm\" (UniqueName: \"kubernetes.io/projected/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-kube-api-access-fzzsm\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.376865 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.376838    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tls-certs\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.376989 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.376968    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-home\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.400867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.400838    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:18.477531 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.477501    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-model-cache\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.477651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.477568    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tmp-dir\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.477651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.477597    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-dshm\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.477651 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.477633    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzzsm\" (UniqueName: \"kubernetes.io/projected/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-kube-api-access-fzzsm\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.477833 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.477690    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tls-certs\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.477833 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.477727    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-home\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.477945 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.477906    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-model-cache\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.478005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.477957    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tmp-dir\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.478090 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.478071    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-home\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.479936 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.479916    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-dshm\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.480102 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.480085    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tls-certs\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.501844 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.501820    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzzsm\" (UniqueName: \"kubernetes.io/projected/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-kube-api-access-fzzsm\") pod \"e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") " pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.633577 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.633493    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:18.940413 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:18.940321    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"]
Oct 30 13:34:18.944322 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:34:18.944283    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5036ca1d_3ff2_40f8_96ec_ad0fb968ba47.slice/crio-c72392030bf39e54231f1b3bacb4eccd29515bd7add06d2c5b447f732e05033a WatchSource:0}: Error finding container c72392030bf39e54231f1b3bacb4eccd29515bd7add06d2c5b447f732e05033a: Status 404 returned error can't find the container with id c72392030bf39e54231f1b3bacb4eccd29515bd7add06d2c5b447f732e05033a
Oct 30 13:34:19.612418 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:19.612383    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw" event={"ID":"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47","Type":"ContainerStarted","Data":"96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899"}
Oct 30 13:34:19.612798 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:19.612427    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw" event={"ID":"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47","Type":"ContainerStarted","Data":"c72392030bf39e54231f1b3bacb4eccd29515bd7add06d2c5b447f732e05033a"}
Oct 30 13:34:19.634307 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:19.634274    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:19.652773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:19.652732    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw" podStartSLOduration=1.6527189770000001 podStartE2EDuration="1.652718977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:34:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:34:19.652326825 +0000 UTC m=+2196.949544367" watchObservedRunningTime="2026-10-30 13:34:19.652718977 +0000 UTC m=+2196.949936514"
Oct 30 13:34:27.708138 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:27.708102    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:28.634396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:28.634356    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:28.646487 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:28.646461    5781 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:28.706833 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:28.706805    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:34:31.995028 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:34:31.994991    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:34:32.594170 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:32.594134    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:32.746869 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:32.746838    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:35.872440 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.872403    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"]
Oct 30 13:34:35.872986 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.872967    5781 cpu_manager.go:414] "containerMap: removing container" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy"
Oct 30 13:34:35.873054 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.872989    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy"
Oct 30 13:34:35.873215 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.873099    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="d55a482c-2288-4209-b9ec-a0539f0958ec" containerName="istio-proxy"
Oct 30 13:34:35.876331 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.876312    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:35.891421 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.891398    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:34:35.903569 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.903544    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:34:35.905972 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.905942    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"]
Oct 30 13:34:35.920070 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.920035    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4pxb\" (UniqueName: \"kubernetes.io/projected/a2e5c857-8e17-482e-97d6-7c336f123640-kube-api-access-v4pxb\") pod \"maas-api-revoke-sub-452b6d7b-j68ts\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:35.920219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.920092    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-j68ts\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:35.920304 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:35.920261    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-j68ts\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:36.021474 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:36.021434    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4pxb\" (UniqueName: \"kubernetes.io/projected/a2e5c857-8e17-482e-97d6-7c336f123640-kube-api-access-v4pxb\") pod \"maas-api-revoke-sub-452b6d7b-j68ts\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:36.021661 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:36.021524    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-j68ts\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:36.021661 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:36.021578    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-j68ts\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:36.022229 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:36.022207    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-j68ts\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:36.023822 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:36.023803    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-j68ts\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:36.045858 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:36.045833    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4pxb\" (UniqueName: \"kubernetes.io/projected/a2e5c857-8e17-482e-97d6-7c336f123640-kube-api-access-v4pxb\") pod \"maas-api-revoke-sub-452b6d7b-j68ts\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:36.189198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:36.189162    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:36.485605 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:36.485508    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"]
Oct 30 13:34:36.493895 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:34:36.493866    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e5c857_8e17_482e_97d6_7c336f123640.slice/crio-d38834be25379528bc4dddd20aa77b3b55601bcb3938950b48f5daaa3d830781 WatchSource:0}: Error finding container d38834be25379528bc4dddd20aa77b3b55601bcb3938950b48f5daaa3d830781: Status 404 returned error can't find the container with id d38834be25379528bc4dddd20aa77b3b55601bcb3938950b48f5daaa3d830781
Oct 30 13:34:36.737438 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:36.737372    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts" event={"ID":"a2e5c857-8e17-482e-97d6-7c336f123640","Type":"ContainerStarted","Data":"d38834be25379528bc4dddd20aa77b3b55601bcb3938950b48f5daaa3d830781"}
Oct 30 13:34:37.744383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:37.744348    5781 generic.go:356] "Generic (PLEG): container finished" podID="a2e5c857-8e17-482e-97d6-7c336f123640" containerID="21fbe5d5c0e3f99b579ffba16d485685546e49db11d0edec50a8b21730e05849" exitCode=0
Oct 30 13:34:37.744867 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:37.744424    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts" event={"ID":"a2e5c857-8e17-482e-97d6-7c336f123640","Type":"ContainerDied","Data":"21fbe5d5c0e3f99b579ffba16d485685546e49db11d0edec50a8b21730e05849"}
Oct 30 13:34:39.202387 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.202356    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:39.284897 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.283980    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-xks-ca\") pod \"a2e5c857-8e17-482e-97d6-7c336f123640\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") "
Oct 30 13:34:39.284897 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.284052    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-openshift-ca\") pod \"a2e5c857-8e17-482e-97d6-7c336f123640\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") "
Oct 30 13:34:39.284897 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.284096    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2e5c857-8e17-482e-97d6-7c336f123640-kube-api-access-v4pxb\" (UniqueName: \"kubernetes.io/projected/a2e5c857-8e17-482e-97d6-7c336f123640-kube-api-access-v4pxb\") pod \"a2e5c857-8e17-482e-97d6-7c336f123640\" (UID: \"a2e5c857-8e17-482e-97d6-7c336f123640\") "
Oct 30 13:34:39.286941 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.286882    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-openshift-ca" pod "a2e5c857-8e17-482e-97d6-7c336f123640" (UID: "a2e5c857-8e17-482e-97d6-7c336f123640"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:34:39.294189 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.294149    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e5c857-8e17-482e-97d6-7c336f123640-kube-api-access-v4pxb" pod "a2e5c857-8e17-482e-97d6-7c336f123640" (UID: "a2e5c857-8e17-482e-97d6-7c336f123640"). InnerVolumeSpecName "kube-api-access-v4pxb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:34:39.294678 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.294646    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-xks-ca" pod "a2e5c857-8e17-482e-97d6-7c336f123640" (UID: "a2e5c857-8e17-482e-97d6-7c336f123640"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:34:39.386364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.385809    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:39.386364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.385844    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a2e5c857-8e17-482e-97d6-7c336f123640-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:39.386364 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.385859    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4pxb\" (UniqueName: \"kubernetes.io/projected/a2e5c857-8e17-482e-97d6-7c336f123640-kube-api-access-v4pxb\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:39.789334 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.785822    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts" event={"ID":"a2e5c857-8e17-482e-97d6-7c336f123640","Type":"ContainerDied","Data":"d38834be25379528bc4dddd20aa77b3b55601bcb3938950b48f5daaa3d830781"}
Oct 30 13:34:39.789334 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.785890    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d38834be25379528bc4dddd20aa77b3b55601bcb3938950b48f5daaa3d830781"
Oct 30 13:34:39.789334 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:39.785962    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"
Oct 30 13:34:43.530750 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:43.530676    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:47.355569 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:47.355533    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:48.323874 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:48.323805    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:49.643304 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:49.643263    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"]
Oct 30 13:34:49.711173 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:49.711144    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:53.438769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.438731    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"]
Oct 30 13:34:53.439761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.439407    5781 cpu_manager.go:414] "containerMap: removing container" podUID="a2e5c857-8e17-482e-97d6-7c336f123640" containerName="revoke-keys"
Oct 30 13:34:53.439761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.439427    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2e5c857-8e17-482e-97d6-7c336f123640" containerName="revoke-keys"
Oct 30 13:34:53.439761 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.439542    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2e5c857-8e17-482e-97d6-7c336f123640" containerName="revoke-keys"
Oct 30 13:34:53.444323 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.444273    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.458978 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.458954    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:34:53.469612 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.469591    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:34:53.472223 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.472193    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"]
Oct 30 13:34:53.544690 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.544657    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-5hb95\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.544857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.544741    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8jvv\" (UniqueName: \"kubernetes.io/projected/46377cde-6c06-4896-b2bb-d776553a684b-kube-api-access-q8jvv\") pod \"maas-api-revoke-sub-3e31a506-5hb95\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.544857 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.544773    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-5hb95\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.645615 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.645574    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-5hb95\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.645818 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.645635    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8jvv\" (UniqueName: \"kubernetes.io/projected/46377cde-6c06-4896-b2bb-d776553a684b-kube-api-access-q8jvv\") pod \"maas-api-revoke-sub-3e31a506-5hb95\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.645818 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.645664    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-5hb95\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.646539 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.646510    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-5hb95\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.648108 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.648079    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-5hb95\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.670863 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.670828    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8jvv\" (UniqueName: \"kubernetes.io/projected/46377cde-6c06-4896-b2bb-d776553a684b-kube-api-access-q8jvv\") pod \"maas-api-revoke-sub-3e31a506-5hb95\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:53.759878 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:53.759792    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:54.078346 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:54.078199    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"]
Oct 30 13:34:54.083370 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:34:54.083335    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46377cde_6c06_4896_b2bb_d776553a684b.slice/crio-2c1aaa8bc46fda4f72cfff3f6f0bb9772a637d4d31e3be065b38aa7dbe9f3bbf WatchSource:0}: Error finding container 2c1aaa8bc46fda4f72cfff3f6f0bb9772a637d4d31e3be065b38aa7dbe9f3bbf: Status 404 returned error can't find the container with id 2c1aaa8bc46fda4f72cfff3f6f0bb9772a637d4d31e3be065b38aa7dbe9f3bbf
Oct 30 13:34:54.906576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:54.906492    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95" event={"ID":"46377cde-6c06-4896-b2bb-d776553a684b","Type":"ContainerStarted","Data":"1f1f7d661256642f7bdc0073e4bfa38d9cca410a5feb364a48b35f39a811cb4a"}
Oct 30 13:34:54.906576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:54.906547    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95" event={"ID":"46377cde-6c06-4896-b2bb-d776553a684b","Type":"ContainerStarted","Data":"2c1aaa8bc46fda4f72cfff3f6f0bb9772a637d4d31e3be065b38aa7dbe9f3bbf"}
Oct 30 13:34:55.917311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:55.913121    5781 generic.go:356] "Generic (PLEG): container finished" podID="46377cde-6c06-4896-b2bb-d776553a684b" containerID="1f1f7d661256642f7bdc0073e4bfa38d9cca410a5feb364a48b35f39a811cb4a" exitCode=0
Oct 30 13:34:55.917311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:55.913173    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95" event={"ID":"46377cde-6c06-4896-b2bb-d776553a684b","Type":"ContainerDied","Data":"1f1f7d661256642f7bdc0073e4bfa38d9cca410a5feb364a48b35f39a811cb4a"}
Oct 30 13:34:56.835252 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:56.835225    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:34:57.054153 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.054128    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:57.108562 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.108490    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46377cde-6c06-4896-b2bb-d776553a684b-kube-api-access-q8jvv\" (UniqueName: \"kubernetes.io/projected/46377cde-6c06-4896-b2bb-d776553a684b-kube-api-access-q8jvv\") pod \"46377cde-6c06-4896-b2bb-d776553a684b\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") "
Oct 30 13:34:57.108562 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.108548    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-xks-ca\") pod \"46377cde-6c06-4896-b2bb-d776553a684b\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") "
Oct 30 13:34:57.108772 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.108630    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-openshift-ca\") pod \"46377cde-6c06-4896-b2bb-d776553a684b\" (UID: \"46377cde-6c06-4896-b2bb-d776553a684b\") "
Oct 30 13:34:57.109040 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.109015    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-openshift-ca" pod "46377cde-6c06-4896-b2bb-d776553a684b" (UID: "46377cde-6c06-4896-b2bb-d776553a684b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:34:57.110658 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.110631    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-xks-ca" pod "46377cde-6c06-4896-b2bb-d776553a684b" (UID: "46377cde-6c06-4896-b2bb-d776553a684b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:34:57.110748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.110632    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46377cde-6c06-4896-b2bb-d776553a684b-kube-api-access-q8jvv" pod "46377cde-6c06-4896-b2bb-d776553a684b" (UID: "46377cde-6c06-4896-b2bb-d776553a684b"). InnerVolumeSpecName "kube-api-access-q8jvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:34:57.209604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.209567    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:57.209604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.209595    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8jvv\" (UniqueName: \"kubernetes.io/projected/46377cde-6c06-4896-b2bb-d776553a684b-kube-api-access-q8jvv\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:57.209604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.209604    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/46377cde-6c06-4896-b2bb-d776553a684b-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:34:57.924426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.924396    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"
Oct 30 13:34:57.924629 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.924427    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95" event={"ID":"46377cde-6c06-4896-b2bb-d776553a684b","Type":"ContainerDied","Data":"2c1aaa8bc46fda4f72cfff3f6f0bb9772a637d4d31e3be065b38aa7dbe9f3bbf"}
Oct 30 13:34:57.924629 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:34:57.924475    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c1aaa8bc46fda4f72cfff3f6f0bb9772a637d4d31e3be065b38aa7dbe9f3bbf"
Oct 30 13:35:01.995305 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:35:01.995258    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:35:04.665855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.665795    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw" podUID="5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" containerName="main" containerID="cri-o://96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899" gracePeriod=45
Oct 30 13:35:04.696787 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.696754    5781 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw" podUID="5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" containerName="main" probeResult="failure" output="Get \"https://10.134.0.139:8000/ready\": dial tcp 10.134.0.139:8000: connect: connection refused"
Oct 30 13:35:04.915239 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.915208    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:35:04.961911 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.961880    5781 generic.go:356] "Generic (PLEG): container finished" podID="5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" containerID="96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899" exitCode=0
Oct 30 13:35:04.962068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.961929    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"
Oct 30 13:35:04.962068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.961958    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw" event={"ID":"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47","Type":"ContainerDied","Data":"96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899"}
Oct 30 13:35:04.962068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.962006    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw" event={"ID":"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47","Type":"ContainerDied","Data":"c72392030bf39e54231f1b3bacb4eccd29515bd7add06d2c5b447f732e05033a"}
Oct 30 13:35:04.962068 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.962028    5781 scope.go:122] "RemoveContainer" containerID="96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899"
Oct 30 13:35:04.972047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.972004    5781 scope.go:122] "RemoveContainer" containerID="96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899"
Oct 30 13:35:04.972324 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:35:04.972275    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899\": container with ID starting with 96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899 not found: ID does not exist" containerID="96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899"
Oct 30 13:35:04.972411 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.972338    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899"} err="failed to get container status \"96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899\": rpc error: code = NotFound desc = could not find container \"96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899\": container with ID starting with 96283639b0a0218e63e0649929bc44c3321ef1f06d9a416e90c382ca7be35899 not found: ID does not exist"
Oct 30 13:35:04.992977 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.992954    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tmp-dir\") pod \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") "
Oct 30 13:35:04.993092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993031    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-dshm\") pod \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") "
Oct 30 13:35:04.993092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993051    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tls-certs\" (UniqueName: \"kubernetes.io/secret/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tls-certs\") pod \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") "
Oct 30 13:35:04.993092 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993088    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-model-cache\") pod \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") "
Oct 30 13:35:04.993261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993119    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-home\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-home\") pod \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") "
Oct 30 13:35:04.993261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993164    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-kube-api-access-fzzsm\" (UniqueName: \"kubernetes.io/projected/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-kube-api-access-fzzsm\") pod \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\" (UID: \"5036ca1d-3ff2-40f8-96ec-ad0fb968ba47\") "
Oct 30 13:35:04.993261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993235    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tmp-dir" pod "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" (UID: "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:35:04.993457 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993372    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-model-cache" pod "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" (UID: "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:35:04.993457 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993416    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-home" pod "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" (UID: "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:35:04.993608 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993588    5781 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tmp-dir\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:04.993722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993612    5781 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-model-cache\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:04.993722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.993627    5781 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-home\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:04.995414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.995374    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-kube-api-access-fzzsm" pod "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" (UID: "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47"). InnerVolumeSpecName "kube-api-access-fzzsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:35:04.995508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.995464    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-dshm" pod "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" (UID: "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:35:04.995580 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:04.995555    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tls-certs" pod "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" (UID: "5036ca1d-3ff2-40f8-96ec-ad0fb968ba47"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:35:05.094769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:05.094741    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzzsm\" (UniqueName: \"kubernetes.io/projected/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-kube-api-access-fzzsm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:05.094769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:05.094767    5781 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-dshm\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:05.094769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:05.094780    5781 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47-tls-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:05.349312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:05.338973    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"]
Oct 30 13:35:05.349312 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:05.339030    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-0b6c7bb2-kserve-844897cb85-wbrhw"]
Oct 30 13:35:06.845861 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:06.845814    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" path="/var/lib/kubelet/pods/5036ca1d-3ff2-40f8-96ec-ad0fb968ba47/volumes"
Oct 30 13:35:10.997802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:10.997748    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:35:13.486383 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:13.486350    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:35:13.657958 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:13.657922    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:35:23.217629 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.217596    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:35:23.896308 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.896250    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"]
Oct 30 13:35:23.896802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.896782    5781 cpu_manager.go:414] "containerMap: removing container" podUID="46377cde-6c06-4896-b2bb-d776553a684b" containerName="revoke-keys"
Oct 30 13:35:23.896802 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.896802    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46377cde-6c06-4896-b2bb-d776553a684b" containerName="revoke-keys"
Oct 30 13:35:23.896973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.896868    5781 cpu_manager.go:414] "containerMap: removing container" podUID="5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" containerName="main"
Oct 30 13:35:23.896973 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.896877    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" containerName="main"
Oct 30 13:35:23.897073 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.896979    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="46377cde-6c06-4896-b2bb-d776553a684b" containerName="revoke-keys"
Oct 30 13:35:23.897073 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.897003    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="5036ca1d-3ff2-40f8-96ec-ad0fb968ba47" containerName="main"
Oct 30 13:35:23.900369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.900347    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:23.925518 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.925495    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 13:35:23.925662 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.925641    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-p9z57\""
Oct 30 13:35:23.927826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:23.927804    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"]
Oct 30 13:35:24.013262 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.013231    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76132619-q245p\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.013426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.013373    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76132619-q245p\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.013481 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.013438    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkcsx\" (UniqueName: \"kubernetes.io/projected/7288918d-c459-47bc-a40b-0082cf4baab3-kube-api-access-pkcsx\") pod \"maas-api-revoke-sub-76132619-q245p\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.114424 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.114396    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76132619-q245p\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.114549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.114463    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkcsx\" (UniqueName: \"kubernetes.io/projected/7288918d-c459-47bc-a40b-0082cf4baab3-kube-api-access-pkcsx\") pod \"maas-api-revoke-sub-76132619-q245p\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.114549 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.114509    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76132619-q245p\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.115042 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.115021    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-76132619-q245p\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.116772 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.116747    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-76132619-q245p\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.150190 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.150143    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkcsx\" (UniqueName: \"kubernetes.io/projected/7288918d-c459-47bc-a40b-0082cf4baab3-kube-api-access-pkcsx\") pod \"maas-api-revoke-sub-76132619-q245p\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.212803 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.212781    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:24.511219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.511188    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"]
Oct 30 13:35:24.516844 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:35:24.516815    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7288918d_c459_47bc_a40b_0082cf4baab3.slice/crio-61e31a0971669787fac7b6a8f8787e69fa0662c2a3fa65a79e500783a62c1836 WatchSource:0}: Error finding container 61e31a0971669787fac7b6a8f8787e69fa0662c2a3fa65a79e500783a62c1836: Status 404 returned error can't find the container with id 61e31a0971669787fac7b6a8f8787e69fa0662c2a3fa65a79e500783a62c1836
Oct 30 13:35:24.670862 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:24.670836    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:35:25.092251 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:25.092175    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p" event={"ID":"7288918d-c459-47bc-a40b-0082cf4baab3","Type":"ContainerStarted","Data":"c0bd0f88388bbd673fe9ec79ec116292cb9811ee4ad8af96d56b582249ce6c68"}
Oct 30 13:35:25.092251 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:25.092224    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p" event={"ID":"7288918d-c459-47bc-a40b-0082cf4baab3","Type":"ContainerStarted","Data":"61e31a0971669787fac7b6a8f8787e69fa0662c2a3fa65a79e500783a62c1836"}
Oct 30 13:35:25.132576 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:25.132531    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p" podStartSLOduration=2.132518339 podStartE2EDuration="2.132518339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:35:25.131820269 +0000 UTC m=+2262.429037810" watchObservedRunningTime="2026-10-30 13:35:25.132518339 +0000 UTC m=+2262.429735877"
Oct 30 13:35:26.099475 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:26.099436    5781 generic.go:356] "Generic (PLEG): container finished" podID="7288918d-c459-47bc-a40b-0082cf4baab3" containerID="c0bd0f88388bbd673fe9ec79ec116292cb9811ee4ad8af96d56b582249ce6c68" exitCode=0
Oct 30 13:35:26.099890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:26.099524    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p" event={"ID":"7288918d-c459-47bc-a40b-0082cf4baab3","Type":"ContainerDied","Data":"c0bd0f88388bbd673fe9ec79ec116292cb9811ee4ad8af96d56b582249ce6c68"}
Oct 30 13:35:27.232386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.232361    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:27.242473 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.242446    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-xks-ca\") pod \"7288918d-c459-47bc-a40b-0082cf4baab3\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") "
Oct 30 13:35:27.242592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.242498    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7288918d-c459-47bc-a40b-0082cf4baab3-kube-api-access-pkcsx\" (UniqueName: \"kubernetes.io/projected/7288918d-c459-47bc-a40b-0082cf4baab3-kube-api-access-pkcsx\") pod \"7288918d-c459-47bc-a40b-0082cf4baab3\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") "
Oct 30 13:35:27.242592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.242547    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-openshift-ca\") pod \"7288918d-c459-47bc-a40b-0082cf4baab3\" (UID: \"7288918d-c459-47bc-a40b-0082cf4baab3\") "
Oct 30 13:35:27.242952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.242931    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-openshift-ca" pod "7288918d-c459-47bc-a40b-0082cf4baab3" (UID: "7288918d-c459-47bc-a40b-0082cf4baab3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:35:27.244426 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.244399    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-xks-ca" pod "7288918d-c459-47bc-a40b-0082cf4baab3" (UID: "7288918d-c459-47bc-a40b-0082cf4baab3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:35:27.244526 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.244488    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7288918d-c459-47bc-a40b-0082cf4baab3-kube-api-access-pkcsx" pod "7288918d-c459-47bc-a40b-0082cf4baab3" (UID: "7288918d-c459-47bc-a40b-0082cf4baab3"). InnerVolumeSpecName "kube-api-access-pkcsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:35:27.343274 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.343246    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkcsx\" (UniqueName: \"kubernetes.io/projected/7288918d-c459-47bc-a40b-0082cf4baab3-kube-api-access-pkcsx\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:27.343274 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.343270    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-openshift-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:27.343274 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.343280    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7288918d-c459-47bc-a40b-0082cf4baab3-maas-api-xks-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:27.786161 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:27.786126    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:35:28.111204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:28.111130    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"
Oct 30 13:35:28.111204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:28.111142    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p" event={"ID":"7288918d-c459-47bc-a40b-0082cf4baab3","Type":"ContainerDied","Data":"61e31a0971669787fac7b6a8f8787e69fa0662c2a3fa65a79e500783a62c1836"}
Oct 30 13:35:28.111204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:28.111184    5781 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61e31a0971669787fac7b6a8f8787e69fa0662c2a3fa65a79e500783a62c1836"
Oct 30 13:35:31.995481 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:35:31.995447    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:35:36.184673 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.184638    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"]
Oct 30 13:35:36.185469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.185203    5781 cpu_manager.go:414] "containerMap: removing container" podUID="7288918d-c459-47bc-a40b-0082cf4baab3" containerName="revoke-keys"
Oct 30 13:35:36.185469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.185228    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7288918d-c459-47bc-a40b-0082cf4baab3" containerName="revoke-keys"
Oct 30 13:35:36.185469 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.185405    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="7288918d-c459-47bc-a40b-0082cf4baab3" containerName="revoke-keys"
Oct 30 13:35:36.188721 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.188702    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.219917 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.219888    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 13:35:36.220113 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.220042    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5qrbm\""
Oct 30 13:35:36.220113 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.220082    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 13:35:36.222184 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.222158    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"]
Oct 30 13:35:36.227677 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.227653    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-maas-api-tls\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.227794 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.227691    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj6ws\" (UniqueName: \"kubernetes.io/projected/97242e9d-24f8-477f-93ec-6bd65a308def-kube-api-access-cj6ws\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.227794 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.227779    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97242e9d-24f8-477f-93ec-6bd65a308def-openshift-service-ca\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.227890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.227829    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-metrics-certs\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.329175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.329130    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-maas-api-tls\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.329175 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.329186    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj6ws\" (UniqueName: \"kubernetes.io/projected/97242e9d-24f8-477f-93ec-6bd65a308def-kube-api-access-cj6ws\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.329464 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.329228    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97242e9d-24f8-477f-93ec-6bd65a308def-openshift-service-ca\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.329464 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.329252    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-metrics-certs\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.330075 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.330045    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97242e9d-24f8-477f-93ec-6bd65a308def-openshift-service-ca\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.331679 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.331650    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-metrics-certs\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.331794 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.331768    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-maas-api-tls\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.352113 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.352085    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj6ws\" (UniqueName: \"kubernetes.io/projected/97242e9d-24f8-477f-93ec-6bd65a308def-kube-api-access-cj6ws\") pod \"maas-api-5c48877c75-kfs5p\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.503405 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.503318    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:36.812142 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.811853    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"]
Oct 30 13:35:36.815823 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:35:36.815787    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97242e9d_24f8_477f_93ec_6bd65a308def.slice/crio-72934f9863b1e3a23e292a7092bfecf90a2739259aa37d79b12be64be7262350 WatchSource:0}: Error finding container 72934f9863b1e3a23e292a7092bfecf90a2739259aa37d79b12be64be7262350: Status 404 returned error can't find the container with id 72934f9863b1e3a23e292a7092bfecf90a2739259aa37d79b12be64be7262350
Oct 30 13:35:36.817075 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:36.817056    5781 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:35:37.167177 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:37.167150    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p" event={"ID":"97242e9d-24f8-477f-93ec-6bd65a308def","Type":"ContainerStarted","Data":"72934f9863b1e3a23e292a7092bfecf90a2739259aa37d79b12be64be7262350"}
Oct 30 13:35:38.173198 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:38.173161    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p" event={"ID":"97242e9d-24f8-477f-93ec-6bd65a308def","Type":"ContainerStarted","Data":"4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915"}
Oct 30 13:35:38.173710 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:38.173309    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:38.217598 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:38.217556    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p" podStartSLOduration=1.882284831 podStartE2EDuration="2.217543007s" totalImagesPullingTime="335.258176ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:35:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:35:38.217204376 +0000 UTC m=+2275.514421917" watchObservedRunningTime="2026-10-30 13:35:38.217543007 +0000 UTC m=+2275.514760540"
Oct 30 13:35:43.619637 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:43.619597    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:35:44.183228 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.183201    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:44.301496 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.301455    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"]
Oct 30 13:35:44.301776 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.301750    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p" podUID="97242e9d-24f8-477f-93ec-6bd65a308def" containerName="maas-api" containerID="cri-o://4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915" gracePeriod=30
Oct 30 13:35:44.334196 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.334170    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-px49g"]
Oct 30 13:35:44.338447 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.338421    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.361224 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.361190    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-px49g"]
Oct 30 13:35:44.414634 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.414609    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb82fb3b-b97f-4172-b62c-2889c5435e08-metrics-certs\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.414766 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.414657    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb82fb3b-b97f-4172-b62c-2889c5435e08-openshift-service-ca\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.414839 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.414787    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb82fb3b-b97f-4172-b62c-2889c5435e08-maas-api-tls\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.414896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.414835    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvkg8\" (UniqueName: \"kubernetes.io/projected/fb82fb3b-b97f-4172-b62c-2889c5435e08-kube-api-access-mvkg8\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.516088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.516051    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb82fb3b-b97f-4172-b62c-2889c5435e08-maas-api-tls\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.516247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.516111    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvkg8\" (UniqueName: \"kubernetes.io/projected/fb82fb3b-b97f-4172-b62c-2889c5435e08-kube-api-access-mvkg8\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.516247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.516193    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb82fb3b-b97f-4172-b62c-2889c5435e08-metrics-certs\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.516247 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.516221    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb82fb3b-b97f-4172-b62c-2889c5435e08-openshift-service-ca\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.516930 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.516908    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fb82fb3b-b97f-4172-b62c-2889c5435e08-openshift-service-ca\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.518522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.518501    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fb82fb3b-b97f-4172-b62c-2889c5435e08-maas-api-tls\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.518596 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.518582    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb82fb3b-b97f-4172-b62c-2889c5435e08-metrics-certs\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.539277 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.539256    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvkg8\" (UniqueName: \"kubernetes.io/projected/fb82fb3b-b97f-4172-b62c-2889c5435e08-kube-api-access-mvkg8\") pod \"maas-api-595cc648c-px49g\" (UID: \"fb82fb3b-b97f-4172-b62c-2889c5435e08\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.552119 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.552101    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:44.617128 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.617100    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97242e9d-24f8-477f-93ec-6bd65a308def-kube-api-access-cj6ws\" (UniqueName: \"kubernetes.io/projected/97242e9d-24f8-477f-93ec-6bd65a308def-kube-api-access-cj6ws\") pod \"97242e9d-24f8-477f-93ec-6bd65a308def\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") "
Oct 30 13:35:44.617262 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.617176    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-metrics-certs\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-metrics-certs\") pod \"97242e9d-24f8-477f-93ec-6bd65a308def\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") "
Oct 30 13:35:44.617262 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.617243    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-maas-api-tls\") pod \"97242e9d-24f8-477f-93ec-6bd65a308def\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") "
Oct 30 13:35:44.617356 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.617273    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97242e9d-24f8-477f-93ec-6bd65a308def-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97242e9d-24f8-477f-93ec-6bd65a308def-openshift-service-ca\") pod \"97242e9d-24f8-477f-93ec-6bd65a308def\" (UID: \"97242e9d-24f8-477f-93ec-6bd65a308def\") "
Oct 30 13:35:44.617731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.617702    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97242e9d-24f8-477f-93ec-6bd65a308def-openshift-service-ca" pod "97242e9d-24f8-477f-93ec-6bd65a308def" (UID: "97242e9d-24f8-477f-93ec-6bd65a308def"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:35:44.619246 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.619220    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-metrics-certs" pod "97242e9d-24f8-477f-93ec-6bd65a308def" (UID: "97242e9d-24f8-477f-93ec-6bd65a308def"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:35:44.619376 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.619267    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-maas-api-tls" pod "97242e9d-24f8-477f-93ec-6bd65a308def" (UID: "97242e9d-24f8-477f-93ec-6bd65a308def"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:35:44.619376 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.619279    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97242e9d-24f8-477f-93ec-6bd65a308def-kube-api-access-cj6ws" pod "97242e9d-24f8-477f-93ec-6bd65a308def" (UID: "97242e9d-24f8-477f-93ec-6bd65a308def"). InnerVolumeSpecName "kube-api-access-cj6ws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:35:44.680436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.680409    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:44.718266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.718180    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj6ws\" (UniqueName: \"kubernetes.io/projected/97242e9d-24f8-477f-93ec-6bd65a308def-kube-api-access-cj6ws\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:44.718266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.718213    5781 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-metrics-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:44.718266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.718229    5781 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97242e9d-24f8-477f-93ec-6bd65a308def-maas-api-tls\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:44.718266 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.718243    5781 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97242e9d-24f8-477f-93ec-6bd65a308def-openshift-service-ca\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:35:44.978743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:44.978678    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-px49g"]
Oct 30 13:35:44.982557 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:35:44.982536    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb82fb3b_b97f_4172_b62c_2889c5435e08.slice/crio-fa5d11d22e72d554e1686a16b7017a740471c9d61e9b8afa4e28dc642540a6ae WatchSource:0}: Error finding container fa5d11d22e72d554e1686a16b7017a740471c9d61e9b8afa4e28dc642540a6ae: Status 404 returned error can't find the container with id fa5d11d22e72d554e1686a16b7017a740471c9d61e9b8afa4e28dc642540a6ae
Oct 30 13:35:45.213162 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.213136    5781 generic.go:356] "Generic (PLEG): container finished" podID="97242e9d-24f8-477f-93ec-6bd65a308def" containerID="4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915" exitCode=0
Oct 30 13:35:45.213334 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.213180    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"
Oct 30 13:35:45.213334 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.213219    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p" event={"ID":"97242e9d-24f8-477f-93ec-6bd65a308def","Type":"ContainerDied","Data":"4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915"}
Oct 30 13:35:45.213334 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.213278    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p" event={"ID":"97242e9d-24f8-477f-93ec-6bd65a308def","Type":"ContainerDied","Data":"72934f9863b1e3a23e292a7092bfecf90a2739259aa37d79b12be64be7262350"}
Oct 30 13:35:45.213334 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.213314    5781 scope.go:122] "RemoveContainer" containerID="4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915"
Oct 30 13:35:45.214491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.214466    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g" event={"ID":"fb82fb3b-b97f-4172-b62c-2889c5435e08","Type":"ContainerStarted","Data":"fa5d11d22e72d554e1686a16b7017a740471c9d61e9b8afa4e28dc642540a6ae"}
Oct 30 13:35:45.222110 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.222095    5781 scope.go:122] "RemoveContainer" containerID="4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915"
Oct 30 13:35:45.222387 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:35:45.222368    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915\": container with ID starting with 4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915 not found: ID does not exist" containerID="4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915"
Oct 30 13:35:45.222471 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.222394    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915"} err="failed to get container status \"4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915\": rpc error: code = NotFound desc = could not find container \"4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915\": container with ID starting with 4acda3b333cbd182d8917311a682e1ba8b525a9af06b2420563c612e3cb0f915 not found: ID does not exist"
Oct 30 13:35:45.262361 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.262311    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"]
Oct 30 13:35:45.273769 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.273752    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-kfs5p"]
Oct 30 13:35:45.363122 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.363098    5781 scope.go:122] "RemoveContainer" containerID="40cc57907f86a4d62463aec9a7cacee803d3fbb684586806a017d3737bf858d0"
Oct 30 13:35:45.379427 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:45.379408    5781 scope.go:122] "RemoveContainer" containerID="53415c147af3a4672d2e44bcb0496cd4206248a039796c119baa26af362e2fc6"
Oct 30 13:35:46.221280 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:46.221243    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g" event={"ID":"fb82fb3b-b97f-4172-b62c-2889c5435e08","Type":"ContainerStarted","Data":"2db46f91f08f738f946364568b171076c55b25a6c8f38e1f2ed76dd000183537"}
Oct 30 13:35:46.221732 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:46.221370    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:35:46.264884 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:46.264836    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g" podStartSLOduration=1.916302638 podStartE2EDuration="2.26482525s" totalImagesPullingTime="348.522612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:35:46.262959768 +0000 UTC m=+2283.560177312" watchObservedRunningTime="2026-10-30 13:35:46.26482525 +0000 UTC m=+2283.562042792"
Oct 30 13:35:46.839356 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:46.839327    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97242e9d-24f8-477f-93ec-6bd65a308def" path="/var/lib/kubelet/pods/97242e9d-24f8-477f-93ec-6bd65a308def/volumes"
Oct 30 13:35:48.347730 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:48.347695    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:35:52.232186 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:35:52.232153    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-px49g"
Oct 30 13:36:01.994192 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:36:01.994156    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:36:26.044574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:36:26.044537    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"]
Oct 30 13:36:26.055597 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:36:26.055573    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d576e68b-vfrzs"]
Oct 30 13:36:26.839890 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:36:26.839854    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f21c1f99-19b0-47b1-b35b-e98e704f0788" path="/var/lib/kubelet/pods/f21c1f99-19b0-47b1-b35b-e98e704f0788/volumes"
Oct 30 13:36:31.995797 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:36:31.995757    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:36:43.035369 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:36:43.035341    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"]
Oct 30 13:36:43.046900 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:36:43.046874    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d2bc93bc-kpbfm"]
Oct 30 13:36:44.856841 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:36:44.856795    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcd7986f-13b4-4f5e-9829-d55b29c6f33e" path="/var/lib/kubelet/pods/dcd7986f-13b4-4f5e-9829-d55b29c6f33e/volumes"
Oct 30 13:36:47.607625 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:36:47.607589    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:36:47.751773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:36:47.751743    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:36:58.466415 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:36:58.466279    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:37:01.995178 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:37:01.995139    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:37:31.993922 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:37:31.993889    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:37:37.355400 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:37.355362    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-jtrd7_6546dc9b-551d-4dc0-988b-6a817d811f4a/manager/0.log"
Oct 30 13:37:38.363379 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:38.363346    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d4d4766b-ckkkt_d640d2bd-3fc1-4d16-8aba-67e885e1a5ff/copy-manifests/0.log"
Oct 30 13:37:38.395274 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:38.395246    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d4d4766b-ckkkt_d640d2bd-3fc1-4d16-8aba-67e885e1a5ff/manager/0.log"
Oct 30 13:37:38.589513 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:38.589481    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-pns2g_dba49010-425f-4862-80ab-887a9930b165/proxy/0.log"
Oct 30 13:37:39.419005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:39.418960    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-px49g_fb82fb3b-b97f-4172-b62c-2889c5435e08/maas-api/0.log"
Oct 30 13:37:40.659204 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:40.659164    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:37:43.150311 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:43.150260    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:37:43.154912 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:43.154883    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nzs4p_162db930-0919-4366-ac55-b8e647cb714b/ovn-acl-logging/0.log"
Oct 30 13:37:43.179148 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:43.179124    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:37:45.499569 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:45.499537    5781 scope.go:122] "RemoveContainer" containerID="0a29ad06f094e2db7a8d4f51bc9a5db64bce4a4adc2c9471cf0c957cdf3236f1"
Oct 30 13:37:45.508045 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:45.508028    5781 scope.go:122] "RemoveContainer" containerID="32988163dfb30472b532267da9679f5eb9d9a9d9b261e48a1749fc8d5efd1b81"
Oct 30 13:37:48.755046 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:48.755015    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:37:50.053382 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:50.053347    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-5hb95_46377cde-6c06-4896-b2bb-d776553a684b/revoke-keys/0.log"
Oct 30 13:37:50.245199 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:50.245176    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-452b6d7b-j68ts_a2e5c857-8e17-482e-97d6-7c336f123640/revoke-keys/0.log"
Oct 30 13:37:50.640028 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:50.639999    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-76132619-q245p_7288918d-c459-47bc-a40b-0082cf4baab3/revoke-keys/0.log"
Oct 30 13:37:50.836777 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:50.836748    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-c4701c16-bdk56_c2333b40-c133-413f-9146-481564ce441c/revoke-keys/0.log"
Oct 30 13:37:58.572578 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:37:58.572546    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:38:01.238429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:01.238384    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-c5ntv_c522dc11-5cd0-4b4a-8003-1400d800d815/postgres/0.log"
Oct 30 13:38:01.993811 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:38:01.993778    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:38:05.031394 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:05.031359    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"]
Oct 30 13:38:05.042587 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:05.042562    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c4701c16-bdk56"]
Oct 30 13:38:05.123399 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:05.123360    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:38:06.840191 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:06.840159    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2333b40-c133-413f-9146-481564ce441c" path="/var/lib/kubelet/pods/c2333b40-c133-413f-9146-481564ce441c/volumes"
Oct 30 13:38:13.323775 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:13.323738    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-wsmjn_7566f692-4d21-4772-9e61-bbd0b220ece2/manager/0.log"
Oct 30 13:38:13.520777 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:13.520750    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-mv7mw_4348bdb9-21fe-415c-b4c6-467611c09642/kuadrant-console-plugin/0.log"
Oct 30 13:38:15.554693 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:15.554646    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fdd59bb7-hlxh8_f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a/kube-auth-proxy/0.log"
Oct 30 13:38:16.137017 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:16.136981    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-jhs8w_273ba69d-816e-4281-84f6-77f9c2180692/istio-proxy/0.log"
Oct 30 13:38:26.552357 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:26.552325    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-pmfxx_538b556d-43d4-43e1-9040-224227990f0d/istio-proxy/0.log"
Oct 30 13:38:27.615956 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:27.615926    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-zpz7d_8a8de30d-9574-4bdf-ba11-c99dba512e69/istio-proxy/0.log"
Oct 30 13:38:29.256460 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:29.256428    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7_ba75e294-c7ff-4681-b4a2-f1892bded953/main/0.log"
Oct 30 13:38:29.458508 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:29.458482    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-86tw4_b9e82ea8-32e0-4366-bded-499dba1661ec/main/0.log"
Oct 30 13:38:29.677405 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:29.677369    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-zv422_906db875-0b0a-41be-9286-7e7de5898576/main/0.log"
Oct 30 13:38:30.284790 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:30.284761    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n_74c20097-c1fa-4692-b0f6-5b69f8c6ee31/main/0.log"
Oct 30 13:38:30.488908 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:30.488879    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl_80c5e747-d78e-4007-bbae-5f3e9f47f303/main/0.log"
Oct 30 13:38:31.995594 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:38:31.995560    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:38:48.886804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:48.886767    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:38:53.009302 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:38:53.009258    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:39:01.995083 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:39:01.995041    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:39:02.324726 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:02.324635    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-jtrd7_6546dc9b-551d-4dc0-988b-6a817d811f4a/manager/0.log"
Oct 30 13:39:03.303612 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:03.303581    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d4d4766b-ckkkt_d640d2bd-3fc1-4d16-8aba-67e885e1a5ff/manager/0.log"
Oct 30 13:39:03.347764 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:03.347736    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64d4d4766b-ckkkt_d640d2bd-3fc1-4d16-8aba-67e885e1a5ff/copy-manifests/0.log"
Oct 30 13:39:03.543404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:03.543373    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-pns2g_dba49010-425f-4862-80ab-887a9930b165/proxy/0.log"
Oct 30 13:39:04.121790 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:04.121761    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-px49g_fb82fb3b-b97f-4172-b62c-2889c5435e08/maas-api/0.log"
Oct 30 13:39:14.754078 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:14.754047    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-3e31a506-5hb95_46377cde-6c06-4896-b2bb-d776553a684b/revoke-keys/0.log"
Oct 30 13:39:14.950368 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:14.950340    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-452b6d7b-j68ts_a2e5c857-8e17-482e-97d6-7c336f123640/revoke-keys/0.log"
Oct 30 13:39:15.367817 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:15.367788    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-76132619-q245p_7288918d-c459-47bc-a40b-0082cf4baab3/revoke-keys/0.log"
Oct 30 13:39:15.784804 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:15.784778    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-c5ntv_c522dc11-5cd0-4b4a-8003-1400d800d815/postgres/0.log"
Oct 30 13:39:17.513914 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:17.513884    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-wsmjn_7566f692-4d21-4772-9e61-bbd0b220ece2/manager/0.log"
Oct 30 13:39:17.720686 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:17.720656    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-mv7mw_4348bdb9-21fe-415c-b4c6-467611c09642/kuadrant-console-plugin/0.log"
Oct 30 13:39:29.598771 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:29.598734    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fdd59bb7-hlxh8_f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a/kube-auth-proxy/0.log"
Oct 30 13:39:30.201537 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:30.201510    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-jhs8w_273ba69d-816e-4281-84f6-77f9c2180692/istio-proxy/0.log"
Oct 30 13:39:30.592344 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:30.592254    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-pmfxx_538b556d-43d4-43e1-9040-224227990f0d/istio-proxy/0.log"
Oct 30 13:39:31.598855 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:31.598831    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-c9ffffc7f-zpz7d_8a8de30d-9574-4bdf-ba11-c99dba512e69/istio-proxy/0.log"
Oct 30 13:39:31.996519 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:39:31.996469    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:39:32.744124 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:32.744093    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-8kpw7_ba75e294-c7ff-4681-b4a2-f1892bded953/main/0.log"
Oct 30 13:39:32.953863 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:32.953834    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-86tw4_b9e82ea8-32e0-4366-bded-499dba1661ec/main/0.log"
Oct 30 13:39:33.138442 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:33.138362    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-zv422_906db875-0b0a-41be-9286-7e7de5898576/main/0.log"
Oct 30 13:39:33.756498 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:33.756468    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-r2b2n_74c20097-c1fa-4692-b0f6-5b69f8c6ee31/main/0.log"
Oct 30 13:39:33.963302 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:33.963267    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-l79fl_80c5e747-d78e-4007-bbae-5f3e9f47f303/main/0.log"
Oct 30 13:39:40.053396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:40.053353    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"]
Oct 30 13:39:40.064820 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:40.064790    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-j68ts"]
Oct 30 13:39:40.839672 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:40.839639    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2e5c857-8e17-482e-97d6-7c336f123640" path="/var/lib/kubelet/pods/a2e5c857-8e17-482e-97d6-7c336f123640/volumes"
Oct 30 13:39:45.538874 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:45.538841    5781 scope.go:122] "RemoveContainer" containerID="e627440de812db0674675fdfb60931a994a8f9828b05e2a9d4ca09f31628b51c"
Oct 30 13:39:45.547354 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:45.547334    5781 scope.go:122] "RemoveContainer" containerID="aae943e584b657bbbc6659626683d66603862184785548912172d8a1c7635e48"
Oct 30 13:39:49.009012 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:49.008966    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:39:57.057592 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:57.057418    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"]
Oct 30 13:39:57.068652 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:57.068627    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-5hb95"]
Oct 30 13:39:58.839899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:39:58.839864    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46377cde-6c06-4896-b2bb-d776553a684b" path="/var/lib/kubelet/pods/46377cde-6c06-4896-b2bb-d776553a684b/volumes"
Oct 30 13:40:01.996348 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:40:01.996307    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:40:03.786050 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:40:03.786017    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:40:28.048013 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:40:28.047975    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"]
Oct 30 13:40:28.059343 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:40:28.059318    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-76132619-q245p"]
Oct 30 13:40:28.839560 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:40:28.839518    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7288918d-c459-47bc-a40b-0082cf4baab3" path="/var/lib/kubelet/pods/7288918d-c459-47bc-a40b-0082cf4baab3/volumes"
Oct 30 13:40:31.995807 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:40:31.995766    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:40:36.295782 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:40:36.295747    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:40:37.416218 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:40:37.416182    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:40:45.569773 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:40:45.569742    5781 scope.go:122] "RemoveContainer" containerID="21fbe5d5c0e3f99b579ffba16d485685546e49db11d0edec50a8b21730e05849"
Oct 30 13:40:49.120952 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:40:49.120917    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:40:58.142627 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:40:58.142581    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:41:01.995601 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:41:01.995563    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:41:03.841261 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:41:03.841229    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:41:31.995243 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:41:31.995167    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:41:45.591073 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:41:45.591041    5781 scope.go:122] "RemoveContainer" containerID="c0bd0f88388bbd673fe9ec79ec116292cb9811ee4ad8af96d56b582249ce6c68"
Oct 30 13:41:45.599275 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:41:45.599257    5781 scope.go:122] "RemoveContainer" containerID="1f1f7d661256642f7bdc0073e4bfa38d9cca410a5feb364a48b35f39a811cb4a"
Oct 30 13:41:49.234105 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:41:49.234064    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:41:58.246691 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:41:58.246653    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:42:01.995014 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:42:01.994971    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:42:03.971940 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:03.971905    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:42:04.645386 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:04.645353    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:42:04.883942 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:04.883896    5781 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log" limit=1024
Oct 30 13:42:08.935896 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:08.935863    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gkrqb_c69d9666-d791-48fe-aad3-7a91f2ab2d2c/global-pull-secret-syncer/0.log"
Oct 30 13:42:09.447723 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:09.447695    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"]
Oct 30 13:42:09.447893 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:09.447870    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy" containerID="cri-o://fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6" gracePeriod=30
Oct 30 13:42:09.933927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:09.933901    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c56xj_39289142-2079-4b45-ba09-8e06aed12276/konnectivity-agent/0.log"
Oct 30 13:42:10.385825 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:10.385747    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-180.ec2.internal_dbd004dde5b9ed65a173b78115c079e0/haproxy/0.log"
Oct 30 13:42:14.701217 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.701188    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:42:14.759983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.759892    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-credential-socket\") pod \"538b556d-43d4-43e1-9040-224227990f0d\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") "
Oct 30 13:42:14.759983 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.759959    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-envoy\") pod \"538b556d-43d4-43e1-9040-224227990f0d\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") "
Oct 30 13:42:14.760235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.759996    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-certs\") pod \"538b556d-43d4-43e1-9040-224227990f0d\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") "
Oct 30 13:42:14.760235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760046    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/538b556d-43d4-43e1-9040-224227990f0d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/538b556d-43d4-43e1-9040-224227990f0d-istiod-ca-cert\") pod \"538b556d-43d4-43e1-9040-224227990f0d\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") "
Oct 30 13:42:14.760235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760074    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-socket\") pod \"538b556d-43d4-43e1-9040-224227990f0d\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") "
Oct 30 13:42:14.760235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760132    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-istio-token\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-istio-token\") pod \"538b556d-43d4-43e1-9040-224227990f0d\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") "
Oct 30 13:42:14.760235 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760166    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/538b556d-43d4-43e1-9040-224227990f0d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/538b556d-43d4-43e1-9040-224227990f0d-istio-podinfo\") pod \"538b556d-43d4-43e1-9040-224227990f0d\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") "
Oct 30 13:42:14.760528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760260    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-kube-api-access-nwjvj\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-kube-api-access-nwjvj\") pod \"538b556d-43d4-43e1-9040-224227990f0d\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") "
Oct 30 13:42:14.760528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760309    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-data\") pod \"538b556d-43d4-43e1-9040-224227990f0d\" (UID: \"538b556d-43d4-43e1-9040-224227990f0d\") "
Oct 30 13:42:14.760528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760162    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-credential-socket" pod "538b556d-43d4-43e1-9040-224227990f0d" (UID: "538b556d-43d4-43e1-9040-224227990f0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.760528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760339    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-certs" pod "538b556d-43d4-43e1-9040-224227990f0d" (UID: "538b556d-43d4-43e1-9040-224227990f0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.760528 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760442    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-socket" pod "538b556d-43d4-43e1-9040-224227990f0d" (UID: "538b556d-43d4-43e1-9040-224227990f0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.760831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760604    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-data" pod "538b556d-43d4-43e1-9040-224227990f0d" (UID: "538b556d-43d4-43e1-9040-224227990f0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.760831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760750    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/538b556d-43d4-43e1-9040-224227990f0d-istiod-ca-cert" pod "538b556d-43d4-43e1-9040-224227990f0d" (UID: "538b556d-43d4-43e1-9040-224227990f0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:42:14.760831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760800    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.760831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760821    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.761047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760834    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.761047 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.760848    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.762984 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.762617    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-envoy" pod "538b556d-43d4-43e1-9040-224227990f0d" (UID: "538b556d-43d4-43e1-9040-224227990f0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:14.762984 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.762681    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-kube-api-access-nwjvj" pod "538b556d-43d4-43e1-9040-224227990f0d" (UID: "538b556d-43d4-43e1-9040-224227990f0d"). InnerVolumeSpecName "kube-api-access-nwjvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.763158 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.763043    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/538b556d-43d4-43e1-9040-224227990f0d-istio-podinfo" pod "538b556d-43d4-43e1-9040-224227990f0d" (UID: "538b556d-43d4-43e1-9040-224227990f0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:42:14.763307 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.763268    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-istio-token" pod "538b556d-43d4-43e1-9040-224227990f0d" (UID: "538b556d-43d4-43e1-9040-224227990f0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:14.861827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.861800    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.861827 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.861825    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/538b556d-43d4-43e1-9040-224227990f0d-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.861987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.861835    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwjvj\" (UniqueName: \"kubernetes.io/projected/538b556d-43d4-43e1-9040-224227990f0d-kube-api-access-nwjvj\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.861987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.861845    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/538b556d-43d4-43e1-9040-224227990f0d-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:14.861987 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:14.861853    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/538b556d-43d4-43e1-9040-224227990f0d-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:15.179746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:15.179713    5781 generic.go:356] "Generic (PLEG): container finished" podID="538b556d-43d4-43e1-9040-224227990f0d" containerID="fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6" exitCode=0
Oct 30 13:42:15.179926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:15.179763    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"
Oct 30 13:42:15.179926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:15.179784    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" event={"ID":"538b556d-43d4-43e1-9040-224227990f0d","Type":"ContainerDied","Data":"fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6"}
Oct 30 13:42:15.179926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:15.179844    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx" event={"ID":"538b556d-43d4-43e1-9040-224227990f0d","Type":"ContainerDied","Data":"8434932f14c7f4ba1b6d95c43a7c698417ca4dd16f8b363ba9b8c37d83249d5e"}
Oct 30 13:42:15.179926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:15.179873    5781 scope.go:122] "RemoveContainer" containerID="fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6"
Oct 30 13:42:15.188675 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:15.188659    5781 scope.go:122] "RemoveContainer" containerID="fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6"
Oct 30 13:42:15.188914 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:42:15.188893    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6\": container with ID starting with fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6 not found: ID does not exist" containerID="fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6"
Oct 30 13:42:15.188959 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:15.188925    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6"} err="failed to get container status \"fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6\": rpc error: code = NotFound desc = could not find container \"fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6\": container with ID starting with fad03caec12a781c42a11c37290456d56e11f3c843a5cd5c4a065937b15a9cf6 not found: ID does not exist"
Oct 30 13:42:15.241486 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:15.241451    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"]
Oct 30 13:42:15.245748 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:15.245725    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pmfxx"]
Oct 30 13:42:16.840145 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:16.840112    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="538b556d-43d4-43e1-9040-224227990f0d" path="/var/lib/kubelet/pods/538b556d-43d4-43e1-9040-224227990f0d/volumes"
Oct 30 13:42:16.870659 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:16.870637    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-wsmjn_7566f692-4d21-4772-9e61-bbd0b220ece2/manager/0.log"
Oct 30 13:42:16.990985 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:16.990966    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-mv7mw_4348bdb9-21fe-415c-b4c6-467611c09642/kuadrant-console-plugin/0.log"
Oct 30 13:42:18.820668 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:18.820641    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9618051c-041c-4f29-8555-01014e22b415/alertmanager/0.log"
Oct 30 13:42:18.954944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:18.954921    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9618051c-041c-4f29-8555-01014e22b415/config-reloader/0.log"
Oct 30 13:42:19.079522 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.079462    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9618051c-041c-4f29-8555-01014e22b415/kube-rbac-proxy-web/0.log"
Oct 30 13:42:19.116664 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.116631    5781 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"]
Oct 30 13:42:19.117491 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.117466    5781 cpu_manager.go:414] "containerMap: removing container" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy"
Oct 30 13:42:19.117588 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.117492    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy"
Oct 30 13:42:19.117588 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.117512    5781 cpu_manager.go:414] "containerMap: removing container" podUID="97242e9d-24f8-477f-93ec-6bd65a308def" containerName="maas-api"
Oct 30 13:42:19.117588 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.117519    5781 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97242e9d-24f8-477f-93ec-6bd65a308def" containerName="maas-api"
Oct 30 13:42:19.117695 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.117653    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="538b556d-43d4-43e1-9040-224227990f0d" containerName="istio-proxy"
Oct 30 13:42:19.117695 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.117665    5781 memory_manager.go:365] "RemoveStaleState removing state" podUID="97242e9d-24f8-477f-93ec-6bd65a308def" containerName="maas-api"
Oct 30 13:42:19.122452 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.122431    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.140072 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.140054    5781 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sstkl\"/\"default-dockercfg-57nhp\""
Oct 30 13:42:19.151752 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.151734    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sstkl\"/\"openshift-service-ca.crt\""
Oct 30 13:42:19.151836 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.151767    5781 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sstkl\"/\"kube-root-ca.crt\""
Oct 30 13:42:19.154183 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.154163    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"]
Oct 30 13:42:19.201548 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.201524    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9618051c-041c-4f29-8555-01014e22b415/kube-rbac-proxy/0.log"
Oct 30 13:42:19.203826 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.203804    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-sys\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.203905 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.203888    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czzlz\" (UniqueName: \"kubernetes.io/projected/f7a1ff80-3780-445f-bad9-119d1c9496b3-kube-api-access-czzlz\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.203953 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.203922    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-proc\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.203953 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.203941    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-podres\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.204023 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.203961    5781 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-lib-modules\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.304591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.304569    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-sys\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.304708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.304607    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czzlz\" (UniqueName: \"kubernetes.io/projected/f7a1ff80-3780-445f-bad9-119d1c9496b3-kube-api-access-czzlz\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.304708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.304634    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-proc\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.304708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.304652    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-podres\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.304708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.304681    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-sys\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.304895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.304710    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-podres\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.304895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.304716    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-proc\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.304895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.304734    5781 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-lib-modules\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.304895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.304777    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f7a1ff80-3780-445f-bad9-119d1c9496b3-lib-modules\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.316005 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.315989    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9618051c-041c-4f29-8555-01014e22b415/kube-rbac-proxy-metric/0.log"
Oct 30 13:42:19.328255 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.328237    5781 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czzlz\" (UniqueName: \"kubernetes.io/projected/f7a1ff80-3780-445f-bad9-119d1c9496b3-kube-api-access-czzlz\") pod \"perf-node-gather-daemonset-r9gqz\" (UID: \"f7a1ff80-3780-445f-bad9-119d1c9496b3\") " pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.429565 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.429546    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9618051c-041c-4f29-8555-01014e22b415/prom-label-proxy/0.log"
Oct 30 13:42:19.436646 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.436632    5781 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:19.546898 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.546879    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_9618051c-041c-4f29-8555-01014e22b415/init-config-reloader/0.log"
Oct 30 13:42:19.735899 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.735831    5781 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"]
Oct 30 13:42:19.740180 ip-10-0-130-180 kubenswrapper[5781]: W1030 13:42:19.740157    5781 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf7a1ff80_3780_445f_bad9_119d1c9496b3.slice/crio-c9e9538f47a6ce1518dc0dfbbf2e2fd13724aa01d2e8b0591b501e47ac3cf067 WatchSource:0}: Error finding container c9e9538f47a6ce1518dc0dfbbf2e2fd13724aa01d2e8b0591b501e47ac3cf067: Status 404 returned error can't find the container with id c9e9538f47a6ce1518dc0dfbbf2e2fd13724aa01d2e8b0591b501e47ac3cf067
Oct 30 13:42:19.822743 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.822724    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-7l2k2_c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/kube-state-metrics/0.log"
Oct 30 13:42:19.932479 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:19.932457    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-7l2k2_c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/kube-rbac-proxy-main/0.log"
Oct 30 13:42:20.046103 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.046040    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-7l2k2_c420e84e-1a2d-4baa-84cd-b6cabe7ec4d4/kube-rbac-proxy-self/0.log"
Oct 30 13:42:20.164811 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.164793    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6dd9dd5bc9-k4v6k_88f01f9c-02af-4e17-9a80-7f769221c703/metrics-server/0.log"
Oct 30 13:42:20.204614 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.204587    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz" event={"ID":"f7a1ff80-3780-445f-bad9-119d1c9496b3","Type":"ContainerStarted","Data":"9dc0d424b72afc0547656a4ca067804cc0bee661e642f735c9b57a66320a0c52"}
Oct 30 13:42:20.204718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.204630    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz" event={"ID":"f7a1ff80-3780-445f-bad9-119d1c9496b3","Type":"ContainerStarted","Data":"c9e9538f47a6ce1518dc0dfbbf2e2fd13724aa01d2e8b0591b501e47ac3cf067"}
Oct 30 13:42:20.204718 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.204677    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:20.244704 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.244664    5781 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz" podStartSLOduration=1.2446523680000001 podStartE2EDuration="1.244652368s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 13:42:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:42:20.243416791 +0000 UTC m=+2677.540634332" watchObservedRunningTime="2026-10-30 13:42:20.244652368 +0000 UTC m=+2677.541869900"
Oct 30 13:42:20.282942 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.282923    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-cdcx5_b8cc6455-76b2-40a2-aa43-5b516d85d543/monitoring-plugin/0.log"
Oct 30 13:42:20.771667 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.771624    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bkk45_1a451455-21be-4f3a-a56d-8c4026bcf654/node-exporter/0.log"
Oct 30 13:42:20.884791 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.884764    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bkk45_1a451455-21be-4f3a-a56d-8c4026bcf654/kube-rbac-proxy/0.log"
Oct 30 13:42:20.997591 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:20.997570    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bkk45_1a451455-21be-4f3a-a56d-8c4026bcf654/init-textfile/0.log"
Oct 30 13:42:21.474088 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:21.474061    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-jmsqh_6214326d-31ce-4577-bd4b-7997c910a058/kube-rbac-proxy-main/0.log"
Oct 30 13:42:21.588746 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:21.588721    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-jmsqh_6214326d-31ce-4577-bd4b-7997c910a058/kube-rbac-proxy-self/0.log"
Oct 30 13:42:21.712370 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:21.712350    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-jmsqh_6214326d-31ce-4577-bd4b-7997c910a058/openshift-state-metrics/0.log"
Oct 30 13:42:21.842169 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:21.842104    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5848b5af-edfa-4001-9c92-4cfa9f528d32/prometheus/0.log"
Oct 30 13:42:21.954637 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:21.954607    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5848b5af-edfa-4001-9c92-4cfa9f528d32/config-reloader/0.log"
Oct 30 13:42:22.073209 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:22.073187    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5848b5af-edfa-4001-9c92-4cfa9f528d32/thanos-sidecar/0.log"
Oct 30 13:42:22.187688 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:22.187670    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5848b5af-edfa-4001-9c92-4cfa9f528d32/kube-rbac-proxy-web/0.log"
Oct 30 13:42:22.301084 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:22.301061    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5848b5af-edfa-4001-9c92-4cfa9f528d32/kube-rbac-proxy/0.log"
Oct 30 13:42:22.413586 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:22.413563    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5848b5af-edfa-4001-9c92-4cfa9f528d32/kube-rbac-proxy-thanos/0.log"
Oct 30 13:42:22.540673 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:22.540600    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5848b5af-edfa-4001-9c92-4cfa9f528d32/init-config-reloader/0.log"
Oct 30 13:42:22.661907 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:22.661881    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-bq5df_03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/prometheus-operator/0.log"
Oct 30 13:42:22.771574 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:22.771487    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-bq5df_03ee4d93-f69e-4b21-84c9-c3cfce9dcdfa/kube-rbac-proxy/0.log"
Oct 30 13:42:23.027436 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:23.027409    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bf446898-96f59_b49051fc-774c-485d-b50b-2486eaccd550/telemeter-client/0.log"
Oct 30 13:42:23.151869 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:23.151846    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bf446898-96f59_b49051fc-774c-485d-b50b-2486eaccd550/reload/0.log"
Oct 30 13:42:23.265801 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:23.265772    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-bf446898-96f59_b49051fc-774c-485d-b50b-2486eaccd550/kube-rbac-proxy/0.log"
Oct 30 13:42:23.386926 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:23.386862    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dcc9fd45-6nh5c_99f8c0d1-d514-488c-9401-b745e93ad9ac/thanos-query/0.log"
Oct 30 13:42:23.502843 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:23.502815    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dcc9fd45-6nh5c_99f8c0d1-d514-488c-9401-b745e93ad9ac/kube-rbac-proxy-web/0.log"
Oct 30 13:42:23.633542 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:23.633519    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dcc9fd45-6nh5c_99f8c0d1-d514-488c-9401-b745e93ad9ac/kube-rbac-proxy/0.log"
Oct 30 13:42:23.754646 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:23.754622    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dcc9fd45-6nh5c_99f8c0d1-d514-488c-9401-b745e93ad9ac/prom-label-proxy/0.log"
Oct 30 13:42:23.872708 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:23.872685    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dcc9fd45-6nh5c_99f8c0d1-d514-488c-9401-b745e93ad9ac/kube-rbac-proxy-rules/0.log"
Oct 30 13:42:23.994396 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:23.994376    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-dcc9fd45-6nh5c_99f8c0d1-d514-488c-9401-b745e93ad9ac/kube-rbac-proxy-metrics/0.log"
Oct 30 13:42:24.492301 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:24.492266    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"]
Oct 30 13:42:24.492697 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:24.492660    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerName="istio-proxy" containerID="cri-o://daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459" gracePeriod=30
Oct 30 13:42:24.503779 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:24.503752    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"]
Oct 30 13:42:24.503951 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:24.503932    5781 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" podUID="273ba69d-816e-4281-84f6-77f9c2180692" containerName="istio-proxy" containerID="cri-o://a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a" gracePeriod=30
Oct 30 13:42:26.219238 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:26.219206    5781 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sstkl/perf-node-gather-daemonset-r9gqz"
Oct 30 13:42:29.769806 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.769781    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:42:29.772722 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.772705    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:42:29.900658 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900578    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-kube-api-access-w4rc4\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-kube-api-access-w4rc4\") pod \"273ba69d-816e-4281-84f6-77f9c2180692\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") "
Oct 30 13:42:29.900658 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900625    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a8de30d-9574-4bdf-ba11-c99dba512e69-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a8de30d-9574-4bdf-ba11-c99dba512e69-istiod-ca-cert\") pod \"8a8de30d-9574-4bdf-ba11-c99dba512e69\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") "
Oct 30 13:42:29.900658 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900655    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-podinfo\") pod \"8a8de30d-9574-4bdf-ba11-c99dba512e69\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") "
Oct 30 13:42:29.900944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900690    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/273ba69d-816e-4281-84f6-77f9c2180692-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/273ba69d-816e-4281-84f6-77f9c2180692-istio-podinfo\") pod \"273ba69d-816e-4281-84f6-77f9c2180692\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") "
Oct 30 13:42:29.900944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900729    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-credential-socket\") pod \"273ba69d-816e-4281-84f6-77f9c2180692\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") "
Oct 30 13:42:29.900944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900762    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-certs\") pod \"273ba69d-816e-4281-84f6-77f9c2180692\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") "
Oct 30 13:42:29.900944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900802    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-socket\") pod \"8a8de30d-9574-4bdf-ba11-c99dba512e69\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") "
Oct 30 13:42:29.900944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900847    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-data\") pod \"273ba69d-816e-4281-84f6-77f9c2180692\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") "
Oct 30 13:42:29.900944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900878    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/273ba69d-816e-4281-84f6-77f9c2180692-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/273ba69d-816e-4281-84f6-77f9c2180692-istiod-ca-cert\") pod \"273ba69d-816e-4281-84f6-77f9c2180692\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") "
Oct 30 13:42:29.900944 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900934    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-token\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-token\") pod \"8a8de30d-9574-4bdf-ba11-c99dba512e69\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") "
Oct 30 13:42:29.901339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900960    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-envoy\") pod \"273ba69d-816e-4281-84f6-77f9c2180692\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") "
Oct 30 13:42:29.901339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.900985    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-istio-token\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-istio-token\") pod \"273ba69d-816e-4281-84f6-77f9c2180692\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") "
Oct 30 13:42:29.901339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901068    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-certs\") pod \"8a8de30d-9574-4bdf-ba11-c99dba512e69\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") "
Oct 30 13:42:29.901339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901095    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-socket\") pod \"273ba69d-816e-4281-84f6-77f9c2180692\" (UID: \"273ba69d-816e-4281-84f6-77f9c2180692\") "
Oct 30 13:42:29.901339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901118    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-socket" pod "8a8de30d-9574-4bdf-ba11-c99dba512e69" (UID: "8a8de30d-9574-4bdf-ba11-c99dba512e69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:29.901339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901141    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-data\") pod \"8a8de30d-9574-4bdf-ba11-c99dba512e69\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") "
Oct 30 13:42:29.901339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901208    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-credential-socket\") pod \"8a8de30d-9574-4bdf-ba11-c99dba512e69\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") "
Oct 30 13:42:29.901339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901239    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-envoy\") pod \"8a8de30d-9574-4bdf-ba11-c99dba512e69\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") "
Oct 30 13:42:29.901339 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901271    5781 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-kube-api-access-kmpx8\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-kube-api-access-kmpx8\") pod \"8a8de30d-9574-4bdf-ba11-c99dba512e69\" (UID: \"8a8de30d-9574-4bdf-ba11-c99dba512e69\") "
Oct 30 13:42:29.901834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901389    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-credential-socket" pod "273ba69d-816e-4281-84f6-77f9c2180692" (UID: "273ba69d-816e-4281-84f6-77f9c2180692"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:29.901834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901577    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a8de30d-9574-4bdf-ba11-c99dba512e69-istiod-ca-cert" pod "8a8de30d-9574-4bdf-ba11-c99dba512e69" (UID: "8a8de30d-9574-4bdf-ba11-c99dba512e69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:42:29.901834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901254    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/273ba69d-816e-4281-84f6-77f9c2180692-istiod-ca-cert" pod "273ba69d-816e-4281-84f6-77f9c2180692" (UID: "273ba69d-816e-4281-84f6-77f9c2180692"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:42:29.901834 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901816    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-certs" pod "8a8de30d-9574-4bdf-ba11-c99dba512e69" (UID: "8a8de30d-9574-4bdf-ba11-c99dba512e69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:29.902043 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901930    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-data" pod "8a8de30d-9574-4bdf-ba11-c99dba512e69" (UID: "8a8de30d-9574-4bdf-ba11-c99dba512e69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:29.902043 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.901966    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-socket" pod "273ba69d-816e-4281-84f6-77f9c2180692" (UID: "273ba69d-816e-4281-84f6-77f9c2180692"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:29.902188 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.902167    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-credential-socket" pod "8a8de30d-9574-4bdf-ba11-c99dba512e69" (UID: "8a8de30d-9574-4bdf-ba11-c99dba512e69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:29.902188 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.902171    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-data" pod "273ba69d-816e-4281-84f6-77f9c2180692" (UID: "273ba69d-816e-4281-84f6-77f9c2180692"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:29.902430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.902275    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a8de30d-9574-4bdf-ba11-c99dba512e69-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:29.902430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.902324    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:29.902430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.902340    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:29.902430 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.902395    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-certs" pod "273ba69d-816e-4281-84f6-77f9c2180692" (UID: "273ba69d-816e-4281-84f6-77f9c2180692"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:29.903771 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.903738    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-podinfo" pod "8a8de30d-9574-4bdf-ba11-c99dba512e69" (UID: "8a8de30d-9574-4bdf-ba11-c99dba512e69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:42:29.903934 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.903911    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-kube-api-access-w4rc4" pod "273ba69d-816e-4281-84f6-77f9c2180692" (UID: "273ba69d-816e-4281-84f6-77f9c2180692"). InnerVolumeSpecName "kube-api-access-w4rc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:29.904111 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.904090    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/273ba69d-816e-4281-84f6-77f9c2180692-istio-podinfo" pod "273ba69d-816e-4281-84f6-77f9c2180692" (UID: "273ba69d-816e-4281-84f6-77f9c2180692"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:42:29.904219 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.904122    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-kube-api-access-kmpx8" pod "8a8de30d-9574-4bdf-ba11-c99dba512e69" (UID: "8a8de30d-9574-4bdf-ba11-c99dba512e69"). InnerVolumeSpecName "kube-api-access-kmpx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:29.904328 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.904305    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-token" pod "8a8de30d-9574-4bdf-ba11-c99dba512e69" (UID: "8a8de30d-9574-4bdf-ba11-c99dba512e69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:29.904927 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.904905    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-istio-token" pod "273ba69d-816e-4281-84f6-77f9c2180692" (UID: "273ba69d-816e-4281-84f6-77f9c2180692"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:42:29.905404 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.905386    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-envoy" pod "273ba69d-816e-4281-84f6-77f9c2180692" (UID: "273ba69d-816e-4281-84f6-77f9c2180692"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:29.905505 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:29.905487    5781 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-envoy" pod "8a8de30d-9574-4bdf-ba11-c99dba512e69" (UID: "8a8de30d-9574-4bdf-ba11-c99dba512e69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:42:30.003437 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003414    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003442    5781 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003455    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003467    5781 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-credential-socket\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003479    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003491    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmpx8\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-kube-api-access-kmpx8\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003505    5781 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4rc4\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-kube-api-access-w4rc4\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003518    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003531    5781 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/273ba69d-816e-4281-84f6-77f9c2180692-istio-podinfo\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003554 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003545    5781 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-workload-certs\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003558    5781 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-data\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003572    5781 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/273ba69d-816e-4281-84f6-77f9c2180692-istiod-ca-cert\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003584    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a8de30d-9574-4bdf-ba11-c99dba512e69-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003597    5781 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/273ba69d-816e-4281-84f6-77f9c2180692-istio-envoy\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.003831 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.003611    5781 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/273ba69d-816e-4281-84f6-77f9c2180692-istio-token\") on node \"ip-10-0-130-180.ec2.internal\" DevicePath \"\""
Oct 30 13:42:30.250594 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.250564    5781 generic.go:356] "Generic (PLEG): container finished" podID="8a8de30d-9574-4bdf-ba11-c99dba512e69" containerID="daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459" exitCode=0
Oct 30 13:42:30.250731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.250611    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"
Oct 30 13:42:30.250731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.250644    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" event={"ID":"8a8de30d-9574-4bdf-ba11-c99dba512e69","Type":"ContainerDied","Data":"daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459"}
Oct 30 13:42:30.250731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.250704    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d" event={"ID":"8a8de30d-9574-4bdf-ba11-c99dba512e69","Type":"ContainerDied","Data":"64a806c9ebad9aca1bb25fbfe78035495e44c203f6ec7611e3190e64ee3e10f3"}
Oct 30 13:42:30.250731 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.250732    5781 scope.go:122] "RemoveContainer" containerID="daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459"
Oct 30 13:42:30.252029 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.252007    5781 generic.go:356] "Generic (PLEG): container finished" podID="273ba69d-816e-4281-84f6-77f9c2180692" containerID="a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a" exitCode=0
Oct 30 13:42:30.252139 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.252051    5781 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"
Oct 30 13:42:30.252139 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.252069    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" event={"ID":"273ba69d-816e-4281-84f6-77f9c2180692","Type":"ContainerDied","Data":"a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a"}
Oct 30 13:42:30.252139 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.252103    5781 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w" event={"ID":"273ba69d-816e-4281-84f6-77f9c2180692","Type":"ContainerDied","Data":"2df742ddddd4eb6565d8470347edf1ee3612cbeac442f7215a735699b5c0944f"}
Oct 30 13:42:30.259895 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.259878    5781 scope.go:122] "RemoveContainer" containerID="daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459"
Oct 30 13:42:30.260132 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:42:30.260106    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459\": container with ID starting with daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459 not found: ID does not exist" containerID="daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459"
Oct 30 13:42:30.260185 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.260146    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459"} err="failed to get container status \"daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459\": rpc error: code = NotFound desc = could not find container \"daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459\": container with ID starting with daa7b24d8f537f386d03a56b248d4edfdfecb430f09497cdc0bcdfc3f47df459 not found: ID does not exist"
Oct 30 13:42:30.260185 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.260171    5781 scope.go:122] "RemoveContainer" containerID="a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a"
Oct 30 13:42:30.268020 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.268000    5781 scope.go:122] "RemoveContainer" containerID="a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a"
Oct 30 13:42:30.268248 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:42:30.268230    5781 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a\": container with ID starting with a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a not found: ID does not exist" containerID="a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a"
Oct 30 13:42:30.268325 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.268254    5781 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a"} err="failed to get container status \"a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a\": rpc error: code = NotFound desc = could not find container \"a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a\": container with ID starting with a8b36ab74a0a0d591e3aa65263114d7dfe8cf17f3b72d0ac28665d2a94e1566a not found: ID does not exist"
Oct 30 13:42:30.305414 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.305389    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"]
Oct 30 13:42:30.317207 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.317187    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zpz7d"]
Oct 30 13:42:30.342337 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.342308    5781 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"]
Oct 30 13:42:30.347930 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.347874    5781 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-jhs8w"]
Oct 30 13:42:30.839931 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.839861    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="273ba69d-816e-4281-84f6-77f9c2180692" path="/var/lib/kubelet/pods/273ba69d-816e-4281-84f6-77f9c2180692/volumes"
Oct 30 13:42:30.840297 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:30.840265    5781 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a8de30d-9574-4bdf-ba11-c99dba512e69" path="/var/lib/kubelet/pods/8a8de30d-9574-4bdf-ba11-c99dba512e69/volumes"
Oct 30 13:42:31.387604 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:31.387578    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fqj4w_793c2518-45fa-46e0-9a23-766ae40d922b/dns/0.log"
Oct 30 13:42:31.510104 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:31.510079    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fqj4w_793c2518-45fa-46e0-9a23-766ae40d922b/kube-rbac-proxy/0.log"
Oct 30 13:42:31.993828 ip-10-0-130-180 kubenswrapper[5781]: E1030 13:42:31.993796    5781 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qlp5s" containerName="extractor"
Oct 30 13:42:32.455140 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:32.455113    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rlghf_853421d2-ef94-4cbd-b251-1f4d82bfb45e/dns-node-resolver/0.log"
Oct 30 13:42:32.582429 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:32.582394    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fdd59bb7-hlxh8_f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a/kube-auth-proxy/0.log"
Oct 30 13:42:33.816300 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:33.816262    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t7zb4_2926b50e-e07d-445d-bbec-eab0005a59bf/node-ca/0.log"
Oct 30 13:42:36.261272 ip-10-0-130-180 kubenswrapper[5781]: I1030 13:42:36.261241    5781 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75fdd59bb7-hlxh8_f0b8e3b0-3f54-4aa7-9461-2e50a8b0649a/kube-auth-proxy/0.log"